System and method for using a video monitoring system to prevent and manage decubitus ulcers in patients

ABSTRACT

A video monitoring system captures image frames of a patient in various positions. The captured image frames are analyzed by the system for changes in a patient&#39;s position or movement, frames in which the system detects one or both of patient movement and repositioning are retained. The system analyzes an area of interest within each image frame that corresponds to an area in the camera&#39;s view field with the patient. Sequential image frames are compared for motion, only frames without motion, where the patient is still, are analyzed.

CROSS-REFERENCE TO RELATED APPLICATION

This application is a continuation of U.S. patent application entitled,“System and Method for Using a Video Monitoring System to Prevent andManage Decubitus Ulcers in Patients”, having application Ser. No.12/804,774, and filed on Jul. 29, 2010, currently pending.

BACKGROUND OF THE INVENTION

Bed sores are lesions that form in the derma. Bed sores are oftenreferred to as pressure ulcers or decubitus ulcers, sometimesimproperly, as they may also be caused by forces other than purepressure, such as shear and friction forces. These lesions may be causedby a number of different types of tissue forces, 1) pressure, or thecompression of tissues; 2) shear force, where a patient's skin remainsstationary while the deep fascia and skeletal muscle move, usuallydownward due to gravity; and 3) friction, or a force resisting theshearing of skin. The frequency, severity and duration of decubitusulcers may be aggravated by humidity; temperature; age; continence andmedication. While decubitus ulcers may erupt on any part of the body,they are especially prone on portions of the body over bony orcartilaginous areas such as hips, spine, lower back area, shoulders,sacrum, elbows, knees, ankles and heels.

Of the three tissue forces, decubitus ulcers occurring from pressure arethe most prevalent, and some experts insist, are the most easilypreventable and treatable type of wound. Pressure, or compression soresare caused by the weight (or force) of a portion of the body (usuallyproximate to a bone) against a surface. A patient is most at risk offorming or worsening of decubitus ulcers by remaining in a decubitus(recumbent) position for a lengthy period without moving. Essentially,blood circulation to an area is restricted by the pressure exerted onthe tissue, usually located between a bone and the surface of a bed. Anopen lesion can form in as little as two or three hours. After anextended amount of time with decreased tissue perfusion, ischemia occursand can lead to tissue necrosis if left untreated. Obviously, bedriddenpatients, who cannot reposition themselves, are among the mostsusceptible to develop pressure sores on their body.

The National Pressure Ulcer Advisory Panel (NPUAP) has categorizedpressure ulcerations into four distinct stages: Stage I is the mostsuperficial, indicated by non blanchable redness that does not subsideafter pressure is relieved. Stage II is damage in the form of a blisteror abrasion to the epidermis extending into, but no deeper than, thedermis. Stage III involves the full thickness of the skin and may extendinto the subcutaneous tissue layer. Stage IV pressure ulcer is thedeepest, extending into the muscle, tendon or even bone. With higherstages, healing time is prolonged. For instance, while approximately 75%of Stage II ulcers heal within eight weeks, only 62% of Stage IVpressure ulcers ever heal, and only 52% heal within one year. (seeThomas D R, Diebold M R, Eggemeyer L M (2005). “A controlled,randomized, comparative study of a radiant heat bandage on the healingof stage 3-4 pressure ulcers: a pilot study”. J Am Med Dir Assoc 6 (1):46-9. doi:10.1016/j.jamda.2004.12.007. PMID 15871870). Therefore, it isimperative for the patient that: 1) the occurrence of decubitus ulcersbe prevented; and 2) decubitus ulcers that have formed be aggressivelytreated and the treatment and progression of the sores monitored.Furthermore, while it was once accepted that decubitus ulcers form atthe surface of the skin, which first begins to deteriorate and thenproceeds inward toward deep tissue, it is now believed that the ulcersbegin at the deepest tissue level, around the bone, and move outwarduntil they reach the epidermis. Hence, once a bed sore becomes visible,there may be a significant amount of hidden tissue damage.

The Agency for Health Care Policy and Research has promulgated a set ofclinical practice guidelines with recommendations for the prediction,prevention, and early treatment of pressure ulcers in adults and whichprovides a summary of supporting evidence for each recommendation.AHCPR. (1992, May). Panel for the Prediction and Prevention of PressureUlcers in Adults. Pressure Ulcers in Adults: Prediction and Prevention.Clinical Practice Guideline, Number 3. AHCPR Publication No. 92-0047.Rockville, Md.: Agency for Health Care Policy and Research, PublicHealth Service, U.S. Department of Health and Human Services. Level VI:Expert Panel Concensus.

The prevention and treatment of decubitus ulcers had escaped medicalprofessionals for centuries, even though they presented a significantpatient mortality risk, until Doreen Norton, FRCN (1922-2007) usedresearch to demonstrate that the best prevention and treatment fordecubitus ulcers was removing the pressure on the patient by turning thepatient. “Turning” refers to exactly what it sounds like, turning thepatient, or changing the patient's position, to prevent the build-up ofpressure on the skin that can result in the loss of circulation. SinceDoreen Norton's research, turning has been universally accepted as themost important factor in bed sore prevention. Yet, despite its universalacceptance, many healthcare facilities (hospitals and nursing homes)fail to properly implement turning techniques.

Recently, patient decubitus ulcers have become a hotspot for malpracticelitigation primarily due to three factors: sympathetic plaintiffs (oftenbedridden or immobile patients); the demonstrability of the harm(decubitus ulcers are easily memorialized in pictures that conveycredible record of harm to the plaintiff); and finally, it can bereadily demonstrated that the treatment of the vast majority of cases isa simple protocol of evaluating the severity of the sore, implementing aturning regimen and monitoring the results. Juries are rightlyconcluding that the cost of prevention and treatment of decubitus ulcersis very low in comparison to the extreme harm that results from no orineffective treatment. Another complication related to decubitus ulcersis their classification as ‘Never Events’, which are events defined byMedicare and the insurance industry as events that should never happenin a hospital. Because patient decubitus ulcers are considered to becompletely preventable, most insurers will not reimburse healthcarefacilities for the cost of treating patient's decubitus ulcers thaterupted during a patient stay. If the hospital allows a patient todevelop a bed sore, the hospital is responsible for all subsequent carefor that sore.

To that end, some healthcare facilities have implemented specific stepsto reduce the occurrence and severity of bed sores, and hence anypotential costs and liability. Firstly, and perhaps most important is toidentify and document the location(s) and severity of decubitus ulcerson all new admissions. Next, a related step is to encourage the staff tolook for and document bed sores, even in their earliest stages.Moreover, any staff member, regardless of their job, is encouraged toalert physicians and supervisors of patients with bed sores, even theearly signs of bed sores. Patients who are bed bound or whose skin comesinto contact with medical equipment like oxygen masks or tubing deserveadditional attention from the staff. Finally, many healthcare facilitiesalso implement a strict wound prevention protocol for implementing dailyskin checks, a patient repositioning regimen, followed up by intensive,hospital-wide skin checks on a regular basis to assess the effectivenessof the protocol.

Turning, or repositioning a patient, is a relatively uncomplicatedconcept, however properly implementing turning techniques is hard,labor-intensive work.

Consequently, patient turning is often performed haphazardly atirregular intervals, without any objective means for documenting theprocedure, or oversight from the healthcare facility. Ideally, patientturning should be completed at intervals set forth by a physician and inthe manner prescribed by the physician. Moreover, the healthcarefacility should have minimum guidelines for preventing the occurrence ofbed sores without specific patient orders from a physician. Forinstance, the turning of patients at least every two hours is oftenconsidered to be the minimally accepted interval. Depending on thecondition of the patient, turning may be required more frequently.Ideally, the patient turns himself to the correct position at theprescribed interval. However, many bedridden patients cannot accomplishthis task without assistance, so the healthcare facility staff shouldreposition the patient in accordance with directions from the attendingphysician or facility guidelines. Additionally, for patients who spend asignificant amount of time in wheelchairs, the healthcare facility staffperiodically removes the patient from their seat and then repositionsthe patient in the wheelchair, again, at the interval and as directed bythe attending physician.

Implementing bed sore abatement regimen has been a manually intensiveprocedure, as has been protocols for documenting the regimen. Typically,a physician prescribes a particular turning regimen for a specificpatient. Ideally, the bed sore prevention regimen can be integratedseamlessly into the caregiver's routine without any additional patientvisits that might interfere with the routines of other patients. As apractical matter, however, most turning regimens require between threeand eight additional patient visits per twelve hour shift. Because eachpatient assigned to a healthcare provider may have a unique turninginterval (also referred to hereinafter as an intervention interval,healthcare professional intervention interval and nurse interventioninterval), keeping the separate patient turning schedules isproblematic. Thus, conflicts between patients' turning schedules arecommonplace, with the labor-intensive turning procedures usually beingsubservient to more effortless care.

Furthermore, the staff is rarely aware of any unaided patient movementsthat might satisfy the turning protocol. The penchant for postponing oroutright skipping turning procedures in view of seemingly conflictingand less strenuous patient care can be mitigated somewhat by a rigorousbed sore prevention documentation procedure and strict oversight. Whenhealthcare professionals are held strictly accountable for implementingthe regimen, they rarely falter. Of course, oversight is yet anothermanually intensive task that consumes valuable time from supervisorypersonnel. Hence, the documentation of turning and turning intervals isusually an additional responsibility assigned to the same healthcareprofessionals responsible for turning the patient. Obviously, thisencourages fraudulent reporting, especially in situations where thecaregiver is exceptionally busy, overworked or detained with moreserious patient matters.

Recently, there has been a trend to automate scheduling anddocumentation of the bed sore prevention regimen. However, many of theseefforts involve essentially recycling known technology for theprevention of bed sores. U.S. Pat. No. 7,378,975 entitled “Method andApparatus for Mitigating the Risk of Pressure Sores,” issued to Smith,et al. on May 27, 2008 discloses using a pressure sensor (such as apressure-sensitive bed mat) in conjunction with a timer to alerthealthcare professionals to the expiration of a turn interval for thepatient. The use of pressure sensors for assessing a patient's presenceon a bed for other support structures is extremely well known forpatient fall detection. Smith, et al. describe a method that integratespatient movement detection into a fall detection procedure for alertingthe healthcare staff of the expiration of a patient turn interval.Essentially, the patient pressure sensor mat senses the position of apatient by individual electrical contacts within the mat. Initially, apatient's position on the sensor mat is assessed and a turn timer resetfor the turn interval. The turn timer continues to count down to theturn time unless a change is sensed at the pressure sensor mat. Thatchange may involve movement indicative of a turn, movement indicative ofthe patient exiting the bed or movement not indicative of a turn. Oncemovement is sensed it is analyzed to determine if the patient left thebed, If so an alarm sounds and the turn timer is reset. If not, thesystem invokes a subroutine to determine if the movement is significant.A “significant move” is defined as a patient movement to a new locationthat is maintained long enough to re-oxygenate the at risk tissue.Hence, if a patient movement is determined to be to a new location onthe pressure sensitive mat, apparently a second timer is initiated fortesting the time at the new location. If sufficiently long, the turntimer resets the turn interval. If not, the turn timer continuescounting the turn interval. Smith, et al. also suggests the use ofinfrared and ultrasonic motion sensors, as well as a conventional orinfrared video camera for sensing patient movement.

The Smith, et al. invention has two distinct advantages over the priorart. Firstly the turning regimen prescribed for a patient may besatisfied by unaided patient movements, thereby eliminating duplicativeand unnecessary turning. Secondly, the healthcare staff is automaticallyalerted to the expiration of a turn interval, regardless of when or howthe interval commenced. In so doing, the attention of the staff is onlynecessary when turning is actually necessary.

The shortcomings with the prior art is that the detected patientmovement may not actually satisfy the turning protocol, yet the turntimer may reset. Additionally, turning, or patient movement, is notdocumented. Documentation, with the exception of possibly noting theissue of a turn alert, apparently remains the responsibility of the turnstaff. Finally, the Smith, et al. invention is directed to alabor-saving device that, based on certain parameters in the system,postpones caregiver turning.

BRIEF SUMMARY OF THE INVENTION

The present invention relates generally to a surveillance system andmethod for monitoring patients, assessing patient positions anddocumenting patient bed sore procedures implemented by the healthcarefacility. More particularly, the present invention relates to a system,method and software program product for determining a patient's positionfrom changes detected between video frames of a surveillance video.

A video monitoring system captures image frames of a patient in variouspositions. The captured image frames are analyzed by the system forpatient movement and/or changes in a patient's position, Frames in whichthe system detects one or both of patient movement and repositioning areretained. The system analyzes an optional area of interest within eachimage frame that corresponds to an area in the camera's view filed withthe patient. Sequential image frames are compared for motion, onlyframes without motion, where the patient is still, are analyzed.Analysis techniques include selecting a plurality of vertical positionson the area of interest, each vertical position corresponding to a partof the patient body indicative of the patient's position, horizontallyscanning values along the horizontal pixel row at each vertical positionin the area of interest, finding an average value for each verticalposition and identifying a patient interval of the horizontal pixel rowhaving pixel values different from the average and a representativemedian horizontal position of the patient interval for each verticalinterval. Then, comparing those representative horizontal positions froma current image frame to the representative median horizontal positionsfor corresponding vertical positions from a previous image frame todetermine a change in the patient's position. Alternatively, the angularrelationship between the representative horizontal positions arecalculated and compared to a library of angular relationships for knownbody positions. The monitoring system also alerts the healthcareprofessional each time the patient should be repositioned based on apatient repositioning schedule. The video monitoring systemautomatically retains an image frame for each scheduled repositioningevent.

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS

The novel features believed characteristic of the present invention areset forth in the appended claims. The invention itself, however, as wellas a preferred mode of use, further objectives and advantages thereof,will be best understood by reference to the following detaileddescription of an illustrative embodiment when read in conjunction withthe accompanying drawings wherein:

FIG. 1 is a flowchart depicting a procedure for the prevention andtreatment of decubitus ulcers in accordance with the prior art;

FIG. 2 is a diagram of the logical components used in the presentpatient movement monitoring system for the prevention and treatment ofdecubitus ulcers in accordance with still another exemplary embodimentof the present invention;

FIG. 3 is a diagram of an exemplary healthcare facility (HCF) in whichthe present patient movement monitoring system may be implemented inaccordance with the exemplary embodiments of the present invention;

FIG. 4 is a diagram of a flowchart illustrating a process forimplementing a semiautonomous video patient repositioning procedure inaccordance with an exemplary embodiment of the present invention;

FIG. 5 is a flowchart depicting a process for a procedure for preventionand/or treatment of decubitus ulcers using generic video analysis inaccordance with an exemplary embodiment of the present invention;

FIGS. 6A and 6B illustrate a lower level view of a process forimplementing a procedure for prevention and/or treatment of decubitusulcers using generic video analysis in which alert events and decubitusulcer procedure events are illustrated with regard to a current imageframe in accordance with an exemplary embodiment of the presentinvention;

FIG. 7 is a screenshot of a setup page for a procedure for preventionand/or treatment of decubitus ulcers in accordance with an exemplaryembodiment of the present invention;

FIGS. 8A-8E illustrate another setup task that greatly optimizes thepresent invention, that of describing an area of interest within thevideo frame where the patient resides;

FIGS. 9A and 9B depict a flowchart illustrating a method for assessingchanges in the position of a patient using video by evaluating discretezones in accordance with an exemplary embodiment of the presentinvention;

FIGS. 10A and 10B depict a table of image analysis results that may beheld in system memory and a corresponding diagram of the zones inaccordance with other exemplary embodiments of the present invention;

FIGS. 11A and 11B are a flowchart depicting a process for assessing apatient's position by detecting the position and/or orientation offacial features in a video frame in accordance with an exemplaryembodiment of the present invention;

FIGS. 12A-12F are diagrams that graphically illustrate the process fordetecting a patient's position from facial features with the patient'shead in various orientations in accordance with an exemplary embodimentof the present invention;

FIGS. 13A-13H are diagrams that graphically illustrate the process fordetecting a patient's position using geometric element comparisons tothe boundary of the patient's head in accordance with an exemplaryembodiment of the present invention;

FIG. 14 is flowchart depicting a process for assessing a patient'sposition by detecting the positions and orientation of the patient'sshoulders in a video frame in accordance with another exemplaryembodiment of the present invention;

FIGS. 15A-15F are diagrams that graphically illustrate the process forassessing a patient's position by detecting the positions andorientation of the patient's shoulders in a video frame in accordancewith another exemplary embodiment of the present invention;

FIGS. 16A and 16B are flowcharts depicting a process for assessingchanges in a patient's position by detecting parts of a patient's bodyrepresented in a video frame that lie along one of a plurality ofdiscrete horizontal lines in accordance with an exemplary embodiment ofthe present invention;

FIGS. 17A and 17B are flowcharts depicting a process for assessing apatient's position by representing the patient's body, from arepresentation in a video frame, as a patient position curve comprisingline segments oriented from each other at particular orientations thatmay be referenced to a library of patient position angular orientationsin accordance with an exemplary embodiment of the present invention;

FIGS. 18A-18C diagrammatically illustrate the area of interestsuperimposed on the patient's bed, in a video frame, with a plurality ofhorizontal stripes in accordance with an exemplary embodiment of thepresent invention;

FIG. 19 is a table that depicts the horizontal positions of the patientat the five discrete horizontal lines in accordance with an exemplaryembodiment of the present invention;

FIG. 20 is a diagram graphically illustrating a plurality of patientposition representation curves represented by line segments at angularorientations between the five exemplary vertical positions in accordancewith an exemplary embodiment of the present invention;

FIG. 21 is a table that illustrates a library of patient positionsrepresented by a sequence of angular relationships between parts of thebody at five exemplary vertical positions in accordance with anexemplary embodiment of the present invention;

FIGS. 22A, 22B and 22C are three screenshots of the video monitoringinterface in response to different decubitus ulcer procedure events inaccordance with exemplary embodiments of the present invention;

FIG. 23 is a flowchart depicting a method creating video history ofpatient decubitus ulcer procedure events in accordance with exemplaryembodiments of the present invention; and

FIG. 24 is a flowchart depicting a method for compiling the videohistory of patient decubitus ulcer procedure events and presenting thevideo history to HC staff as a chronologically ordered set ofscreenshots of the patient in accordance with an exemplary embodiment ofthe present invention.

Other features of the present invention will be apparent from theaccompanying drawings and from the following detailed description.

DETAILED DESCRIPTION OF THE INVENTION

Element Reference Number Designations 200: Patient movement monitoringsystem 210: Camera control device 211: Processor unit 212: Networkcontroller 213: Video processor 214: Primary nonvolatile memory 215:Secondary nonvolatile memory 216: Video camera 217: Video monitor 218:Medical procedure remote interface 220: Local surveillance sub-system222: Medical procedure remote interface 224: Autonomous sensing device226: Medical procedure/pillow speaker interface 230: Patient room 240:Distribution network 250: HCF (Nurse) station 260: Nurse monitor device261: Processor unit 262: Network controller 263: Video processor 264:Primary nonvolatile memory 265: Secondary nonvolatile memory 266: Videocamera 267: Video monitor/touchpad 268: Audible alarm 269: Manualinterface device 270: Patient monitoring system 310: General purposecomputer 321: Patient bed 341: Transmission medium 342: Network switch343: Broadband connection 344: Network storage 345: Patientadministration 346: Network server/router/firewall 348: Network systemadministration 700: ProcedureView screenshot 701: Procedure comments702: Auto patient position selection 704: Patient alarm selection 706:Patient movement selection 708: Position sample frame time 709: Rollingtime interval selection 710: Patient turn schedule 712: Patientposition/time selections 714: Position cycle summary 720: Date/Time/Room730: Procedure information 740: Patient information 832: Pointer 840:Area of interest 1010: Vertical axis 1012: Horizontal axis

In the following description, reference is made to the accompanyingdrawings that form a part hereof, and in which is shown by way ofillustration, specific embodiments in which the invention may bepracticed. These embodiments are described in sufficient detail toenable those skilled in the art to practice the invention, and it is tobe understood that other embodiments may be utilized. It is also to beunderstood that structural, procedural and system changes may be madewithout departing from the spirit and scope of the present invention.The following description is, therefore, not to be taken in a limitingsense. For clarity of exposition, like features shown in theaccompanying drawings are indicated with like reference numerals andsimilar features as shown in alternate embodiments in the drawings areindicated with similar reference numerals.

For simplicity, certain terms will be used synonymously throughout thespecification and in the claims. For instance, the term “bed sore” willbe used synonymously with pressure sore, pressure ulcer and decubitusulcer even though there may be slight technical differences and medicaldefinitions of the terms. Likewise, for inclusiveness, personnel withresponsibility for or duty to a patient or resident will be referred tointernally as a healthcare facility (HCF) staff, professionals or both,regardless of whether that person's vocation is that of a physician,nurse, therapist, aide (or nurses' aid), administrator or any other typeof healthcare worker that interacts with the patient or resident for thepurposes of providing or facilitating a healthcare service. It should,however, be appreciated that given the nature of certain aspects of thepresent invention, that the HCF staff may not necessarily be present onthe facility campus but may be present at a remote site, such as anoffice for a physician, insurance compliance agent, lawyer, facilitymanager, etc. Additionally, the description of the present invention isbetter understood in relation to certain events that may or may notoccur in accordance with a procedure for the prevention and treatment ofdecubitus ulcers. These events are referred to synonymously, among otherterms, as repositioning events, turn events, decubitus ulcer events,decubitus ulcer procedure events, movement events, motion events, or anyevent that might define an action or omission relating to a procedurefor, prevention of or treatment of decubitus ulcers. These events arememorialized videographically, on individual video frames, sequences ofvideo frames or some combination of the two. While the present inventionwill be described largely with respect to a video frame or frames, asingle video frame may contain additional information such as audio andannotated information, either manually input or derived automatically bythe system, timestamps, etc., or alternatively a group of video framesmay be collected as a file of related data that also contains additionalrelated information, i.e., audio, annotated information, timestamps,etc. Still further, the term “healthcare facility” will be usedsynonymously with any facility in which the patient/resident receivesmedical treatment such as a hospital, clinic, nursing home, long termcare facility (typically a facility for treating specialized ailmentsother than age, such as severely disabled or paraplegic patients,comatose patients, end-of-life patients, etc.) and may in fact extend tothe assisted living facilities and patient's home or residence in casesin which ongoing care is provided in a non-hospital patient/residentsetting (such as for hospice and care for Alzheimer's victims).Importantly, the present invention is directed to reducing and/ortreating bed sores, therefore, the terminology used herein to describethe invention should not limit the intended scope of the invention orcomponents or the function of components used therein.

FIG. 1 is a flowchart depicting a procedure for the prevention andtreatment of decubitus ulcers in accordance with the prior art.Typically, the prior art procedure is largely non-automated and verylabor intensive, usually requiring a healthcare professional or facilitystaff (HCF staff) to make observations of the patient at regularintervals, record observations in the patient record, turn (orreposition) the patient at scheduled time intervals and finally recordthe patient's repositioning. Essentially, the procedure begins with thereceipt of a patient order for the prevention and treatment of decubitusulcers that includes patient positions and a turn schedule for HCF staffinterventions with time intervals corresponding to each unique position(step 102). Generally, a patient is allowed to remain in a particularposition for only a predetermined amount of time, perhaps 50 minutes,and then should be repositioned. An order usually lists a sequence ofpatient positions with the corresponding time in the position, forinstance: Back—30 minutes; Left Quarter—15 minutes; Left Side—15minutes; Right Quarter—15 minutes; and Right Side—15 minutes. In theforegoing example the patient will be repositioned five times over a onehour and thirty minute repositioning cycle schedule. In that case theHCF staff sets the repositioning alert (step 104), for instance 30minutes until the patient should be repositioned from the back to theleft quarter in accordance with the exemplary patient order immediatelyabove. The patient's record is updated with the patient's currentposition, currently on the back, and the time (step 106). The procedurethen moves to the patient monitoring phase where the HCF staffcontinually monitors the patient's position (step 108), checks orverifies that it is not time to reposition the patient or that thepatient has not repositioned himself contrary to the patient order (step110) and records the observations in the patient record. Theseobservations are necessary to ensure that the patient is not turningherself to a position between repositioning times that conflict with theturn schedule of the patient order and, if necessary, repositioning thepatient back to a position specified in the patient order.

The aim of the patient repositioning procedure is to keep the patientoff of parts of the body with existing decubitus ulcers, or wheredecubitus ulcers are prone to erupt, long enough to maintain (orreestablish) healthy blood flow at these sites (e.g., portions of thebody over bony or cartilaginous areas such as hips, spine, lower backarea, shoulders, sacrum, elbows, knees, ankles and heels). If thepatient turns herself prematurely, contrary to the turn schedule, theattending HCF staff should return the patient to the correct position toensure sufficient blood flow to vital tissue. Often a patient isuncomfortable on a quarter or side and will turn themselves to theirback immediately after being turned away from that position by the HCFstaff. This is especially common when the patient's view of visitors,reading materials or television is obstructed due to their position. Asa consequence, if left unchecked the patient may spend substantiallylonger periods of time in the same position, precipitating the eruptionof new or more severe decubitus ulcers.

In any case, the time is continually monitored by the HC professional.After the prescribed time interval has elapsed (step 110), the HCF staffrepositions the patient to the next position in the turn schedule (step112), resets the patient repositioning alert corresponding to the newposition in the schedule (step 104) and updates the patient record toreflect the patient repositioning event and the time (step 106). Theprocedure then iterates through steps 108, 110 and 106 as describedabove until the prescribed time interval at that position has elapsed.The patient is then again repositioned in accordance with the patientorder (step 112), and the alerts and patient record is updated (steps104 and 106). The procedure continues unabated until the patient orderis amended or canceled or the patient is discharged.

As may be appreciated from the forgoing, the prior art procedure for theprevention and treatment of decubitus ulcers is extremely tedious,manually intensive and prone to lapses in protocol and judgment by HCFstaff. As the only record in support of a decubitus ulcer procedurebeing performed correctly is the written patient record, rarely will arecord reflect lapses in the turning protocol. What is needed is apatient monitoring system that provides an unbiased and unassailablerecord of patient decubitus ulcer procedure events. What is needed is anautomated patient bed sore system that independently documents patientmovement and turning, without manual intervention from healthcareprofessionals. The record should be annotatable by authorized healthcareprofessionals. Additionally, the document record produced by theautomated patient bed sore system should be protected from unauthorizedalteration. Optimally, the system should accurately distinguish patientmovement that satisfies the prescribed patient turning protocol frompatient movement that does not. Ideally, the system should not onlytrack healthcare professional intervention intervals, but also determinethe orientation of the patient.

Therefore, in an effort to overcome the shortcoming of the prior art andto aid in the prevention and treatment of decubitus ulcers, a system andmethod for prevention and treatment of decubitus ulcers using a videomonitoring system is presented. As will be described further below, theuse of the present invention significantly decreases the amount ofmanual intervention necessary from the HCF staff by integrating thepatient procedure order for the prevention and treatment of decubitusulcers into the patient surveillance system, thereby, providing a realtime view of the patient, alerting the HCF staff of patient-initiatedrepositioning events, automatically and simultaneously tracking both thetime, elapsed time in a position, the current patient position and thenext patient position as well as automatically alerting the HCF staff toscheduled repositioning events. Additionally, the present system andmethod for prevention and treatment of decubitus ulcers using a videomonitoring system provides the HCF staff and others with a conciserecord of decubitus ulcer procedure events that can be reviewed as asequence of annotated video frames without superfluous and unnecessaryvideo. Finally, the present invention provides an attending HCF staffprofessional with a mechanism for rapidly reviewing the procedure over apredetermined time (perhaps a single repositioning cycle or a timeinterval) in order to quickly assess the state of the patientpositioning in the turn cycle and the adherence to the patient order.

FIG. 2 is a diagram of the logical components used in the presentpatient movement monitoring system for the prevention and treatment ofdecubitus ulcers in accordance with an exemplary embodiment of thepresent invention. Typically, the presently described patient movementmonitoring system 200 is implemented in a patient surveillance network,which usually comprises at least patient surveillance sub-system 220 andpatient monitoring sub-system 270. Both patient surveillance sub-system220 and patient monitoring sub-system 270 are available from CareViewCommunication, Inc. of Lewisville, Tex. As may be appreciated, thepresent patient movement monitoring system may also be implementedacross several physical locations, such as patient room 230 (containingpatient surveillance sub-system 220) and HCF or nurse station 250(containing patient monitoring sub-system 270). The separate sub-systemsmay also be realized in virtually any location in the healthcarefacility, such as the offices for patient administration, billing,medical records and network administration or even at off site locationssuch as at a healthcare provider's office, an insurance complianceagent's office, a lawyer's office, or on a mobile device capable ofaccessing the Internet such as a laptop, PDA, smartphone or other suchnet device, depending on the duties of the particular location. FIG. 3is a diagram of an exemplary healthcare facility (HCF) in which thepresent patient movement monitoring system may be implemented.

The components that are typically located in patient surveillancesub-system 220, such as in patient room 230, include camera controldevice 210 that is usually juxtaposed to television 217, but itsposition is not essential to the practice of the present invention. Inmost patient rooms, television 217 is installed at a central locationcentered with the foot of the patient's bed, which is also a highlyadvantageous viewpoint location for installing surveillance camera 216.In accordance to one aspect of the present invention, surveillancecamera 216 should be aligned lengthwise with the patient's bed, as closeto centered as possible wherein the camera's view axis will equallysubdivide the patient's position in the bed. The view angle with respectto the patient's bed should also be noted and considered during videoprocessing as the vertical dimension of the patient is reduced as theview axis transcends 0 degrees (with the surveillance camera 216positioned directly over the patient's bed) to the usual angle ofapproximately 30 degrees (with the surveillance camera 216 positioned onthe wall in front of the patient's bed). The frontal placement ofsurveillance camera 216 changes the aspect (reduces the size) of thepatient in the video frame by approximately half with respect to theoverhead position (assuming 30 degrees view axis angle). Additionally, amicrophone (not shown) may be disposed on surveillance camera 216 ormedical procedure/pillow speaker interface 226 camera control device 210or connected as a separate peripheral for capturing audio in thesurveillance area. Hence, for many installations, camera control device210, television 217 and surveillance camera 216 are loosely coupledtogether as a unit. In any case, camera control device 210 provides thelocal processing, storage and network connections for the surveillanceperipherals and for the present patient medical procedure documentationsystem. Here it should be mentioned that much of the functionality ofthe present invention may be embodied in a standard personal computer,however, other aspects of the present invention may require supplementalvideo processing and/or storage capacity. Furthermore, as may beappreciated from the description of the set-top box in U.S. Pat. No.7,477,825, issued Jan. 13, 2009, which is incorporated herein byreference in its entirety, and copending U.S. patent application Ser.Nos. 12/151,452 and 12/589,654, camera control device 210 may also haveCATV, Internet, PSTN and other capabilities that are not traditionallyfound in a conventional personal computer.

With further regard to camera control device 210, processor unit 211diagrammatically represents all the processing capacity, RAM and ROMmemory, busses and the physical framework for storing and executinginstructions for operating the other components of the control unit.Network controller 212 provides a connection to healthcare facility(HCF) distribution network 240 and to other devices connected to the HCFnetwork, such as nurse monitor device 260 of patient monitoringsub-system 270. Video processor 211 comprises any video processingcapabilities necessary for capturing, processing, video analysis and/ordisplaying any video and/or patient medical procedure documentationscreens. Video processor 213 may be integrated in a general purposeprocessing system or supplement the video processing capabilities of thegeneral purpose processing system. As such, video processor 211 isresponsible for receiving the captured video frames from video camera216, analyzing video for motion (see U.S. Pat. No. 7,477,825 andcopending U.S. patent application Ser. Nos. 12/151,452 and 12/589,654),recognizing patients, patient areas and patient positions, prioritizingvideo frames based on content or external factors (such as labeling theframes as documentation for a patient medical procedure) and compilingmedical procedure information screens for display on the local monitor,such as TV 217. Additionally, video processor 211 may also be capableof, by working in conjunction with the other subcomponents of cameracontrol device 210, analyzing video image frames and comparing one framewith another for such attributes as detection motion in a frame orbetween frames, detecting patient movement motion in a frame or betweenframes, detecting a patient's position in a frame or between frames,inferring patient movement in a frame or between frames, identifying apatient's position in a frame, detecting patient repositioning in aframe or between frames and detecting, analyzing and identifying othermotion in the patient's room that is not related to the patient.

Camera control device 210 also comprises receiver/interrogator andmedical procedure remote interface 218 for communicating with a medicalprocedure sensing device (a manual or autonomous remote interface forsensing an event indicative of the commencement of a patient medicalprocedure). Optimally, receiver/interrogator and medical procedureremote interface 218 provides multiple communications ports forconnecting with multiple types of medical procedure sensing devices,e.g., autonomous sensing devices 224, medical procedure remote interface222, medical procedure/pillow speaker interface 226 and/or some type oflegacy interface device. The medical procedure remote device may operateautonomously (usually by sensing the presence of the HCF staff throughautonomous sensing devices 224) or manually by receiving manuallyinvoked communication from an HC professional. In either case, the aimis for camera control device 210 to receive supplemental informationindicative of the commencement (and possibly termination) of a patientmedical procedure, such as turning the patient for the preventiontreatment of decubitus ulcers in accordance with a patient order, turnschedule or general HCF ulcer prevention procedures. The receipt of thisinformation enables camera control device 210 to flag any subsequentlycaptured AN data as documentation for the information indicative of apatient medical procedure. Hence, that AN data may be prioritized and/orbacked up locally for access in the future. To that end, camera controldevice 210 comprises at least one nonvolatile memory for temporarilystoring AN data documentation of a patient medical procedure, forinstance the prevention and treatment of patient decubitus ulcers.

As further depicted in FIG. 2, camera control device 210 furthercomprises primary nonvolatile memory 214 and optionally, secondarynonvolatile memory 215 for storing different classes of captured ANdata. Generally, primary nonvolatile memory 214 is the designatedarchival storage from more important or higher priority surveillancedata, while secondary nonvolatile memory 215 is a persistent systemmemory used for and by system processes. Hence, if persistence is notnecessary, faster and cheaper volatile RAM can be substituted for themore expensive and slower flash memory of secondary nonvolatile memory215. The storing operations of camera control device are generallydiscussed in U.S. Pat. No. 7,477,825 and copending U.S. patentapplication Ser. Nos. 12/151,452 and 12/589,654. It should beappreciated, however, that surveillance data received by camera controldevice 210 may comprise varying degrees of importance, and, therefore,treated differently depending on its importance and age. Although theoperation of the present invention will be described throughout withregard to the “single path” approach of handling surveillance data(i.e., AN data comprising surveillance full-motion video and audio, ifpresent), the surveillance data may instead be handled in a “two path”approach. The particular type of memory management and data retentiontechnique is relatively unimportant for the practice of the presentinvention. Either of these approaches, or some other memory managementapproach may be utilized without departing from the scope of thepresently described invention.

Typically, most surveillance data received by camera control device 210is of relatively low importance and, therefore, need not be retained inprimary nonvolatile memory 214. Using the single path approach, thesurveillance data makes a single path from the capture device(s) tosecondary nonvolatile memory 215 and finally, depending on itsimportance, to primary nonvolatile memory 214. Initially, capturedsurveillance data is temporarily retained in secondary nonvolatilememory 215 regardless of its importance for eventual transmission tomonitoring device 260 for temporal monitoring by the HCF staff in nearreal time (such as a nurse at nurse station 250). Surveillance data thatcamera control device 210 recognizes as having a higher priority, suchas documentations of patient medical procedures, is typically archivedto primary nonvolatile memory 214 for retention, downloading and/orreviewing at a later time. Consequently, some of the data in secondarynonvolatile memory 215 is considered to have a relatively low priority.Low priority and older surveillance data in secondary nonvolatile memory25 will be the first data to be overwritten by fresher surveillance datareceived at camera control device 210 (the decision of whichsurveillance data in secondary nonvolatile memory 215 to overwrite maybe based on the results of the application of a weighted algorithm tothe data). Additionally, some surveillance data received by cameracontrol device 210 may be flagged for further review by the HCF staff,thereby increasing its importance, at least temporarily. Thatsurveillance data is also stored in primary nonvolatile memory 214. Thistype of data might include AN data that failed to be immediatelytransmitted over distribution network 240 due to network bandwidth oroperation issues. Various techniques may be applied to this data forachieving a rapid resolution to the problem, such as alarms, frame ratereduction and locally backing up the AN data. Hence, in the single pathapproach, only surveillance data of relatively high importance isarchived in primary nonvolatile memory 214, while conversely, most orall of the surveillance data, regardless of its importance, istemporarily saved in secondary nonvolatile memory 215 to be, forexample, available for system use. The system constantly analyzes theimportance and age of the data in secondary nonvolatile memory 215 forarchival in primary nonvolatile memory 214 and for overwriting withfresher surveillance data (using, for example, a weighted algorithm).Similarly, system also analyzes the importance and age of the dataretained in primary nonvolatile memory 214 whenever additional archivalspace is needed for fresher high priority surveillance data.

Alternatively, using the two-path the surveillance data follows separatepaths through camera control device 210. Initially, surveillance data(i.e., AN data comprising surveillance full-motion video and audio, ifpresent) is recorded directly and continuously into secondarynonvolatile memory 215. The system periodically moves the surveillancedata into the larger primary nonvolatile memory 214, where it archivesthe data indefinitely, or until it reaches the capacity of primarynonvolatile memory 214. Separately, camera control device 210 receivescaptured image frames from surveillance camera 216, and performs imageanalysis on those frames to support detection methods such as discussedbelow (in some cases the video format of the captured surveillance datawill be converted to another format more suitable for video processing,such as from an MPEG specification standard format to a JPEGspecification standard format). The still images and corresponding imagedata processed by the system are kept in a volatile memory (not shown)to allow for efficient processing. As useful patterns or events aredetected through the image analysis, the system persists those eventsinto a database in primary nonvolatile memory 214. The informationpersisted by these events includes at least the system timestamp, anoffset for referencing into the currently recorded video, the type ofevent, any supporting data pertaining to that event, etc. Higherpriority surveillance data in primary nonvolatile memory 214 will beassociated with specific event information. Surveillance data in primarynonvolatile memory 214 without any associated event information has alower priority and is subject to overwriting. At some point in time,usually determined by a weight algorithm, even high prioritysurveillance data is subject to overwriting with fresher and perhapslower priority surveillance data.

On an ongoing basis, the system reviews the events associated withspecific timestamps in the video, and uses a weighted algorithm todetermine which segments of video are least important, and may be purgedfrom the archival data retained in primary nonvolatile memory 214. Thesystem then extracts the portions of the recorded video on primarynonvolatile memory 214 which are the least important as determined bythe weighted algorithm, and deletes them. This allows the presentlydescribed patient movement monitoring system to maximize theeffectiveness of the limited nonvolatile storage available, and retainas much video which would be necessary to review at a later date aspossible.

Other classes of data that should be treated separately are surveillancedata relating to patient medical procedures. One type of surveillancedata is associated with information received by receiver/interrogatorand medical procedure remote interface 218 that is indicative of apatient medical procedure. All surveillance data captured within a timewindow of the receipt of patient medical procedure information may betreated as documentary evidence of that patient medical procedure.Hence, surveillance data retained in secondary nonvolatile memory 215prior to receiving the patient medical procedure information may beconsidered as documentation of a patient medical procedure and copied toprimary nonvolatile memory 214 for more permanent storage.

Another type of data is information derived independently by cameracontrol device 210 is that which is indicative of motion, patientmovement or repositioning and other events that might be applicable tothe prevention and treatment of decubitus ulcers, i.e., at leastpotential motion, patient movement and patient repositioning. This typeof data are flagged for further review and/or verification by the HCFstaff, but is also automatically saved to a secure memory, such asprimary nonvolatile memory 214, a network database, or both, by a systemprocess that is beyond the control of the HCF staff. Hence, videoinformation related to the decubitus ulcer procedure events may beautomatically, accurately and securely retained for documentingprevention and treatment procedures for decubitus ulcers.

Before proceeding, it should be mentioned that surveillance AN data maybe retained in one of several formats. One retention format involvesstoring the sequentially captured image frames as separate image framefiles. Patient procedure information received at patient surveillancesub-system 220 is included in the frame header for corresponding imageframe. This retention format is useful for streaming video across theHCF distribution network; if documentation of a patient medicalprocedure is needed, each of the image frames in storage must besearched and returned separately. The returned AN frames may then becombined into a single surveillance AN documentation file for theprocedure. A second type retention format is storing a group of relatedvideo image frames in a single surveillance AN file. Here, rather thanduplicating the patient procedure information to each frame header, theinformation is saved with the file. For example, patient surveillancedata for a medical procedure residing in secondary nonvolatile memory215 that were captured within a predetermined time window are includedin a single procedure documentation AN file. All NV data captured atpatient surveillance sub-system 220 are bound to the proceduredocumentation AN file until an end-of-procedure event is logged. Theend-of-procedure event may be any information indicative of an end ofthe patient medical procedure, for instance, receiving a manual commandfrom the HCF staff to terminate the medical procedure NV data file,detecting the end of a predetermined time period, or by failing todetect any motion in the surveillance area for a predetermined timeperiod, or any combination of the sub-processes described above. Withfurther regard to either retention format, the aim is to createdocumentation of a patient medical procedure as a concise sequential,although not necessarily continuous, stream of surveillance data thatcan be easily identified as corresponding to a particular patientmedical procedure.

It should be mentioned that, in some situations patient surveillancesub-system 220 automatically identifies video frames associated with aparticular patient procedure without manual intervention from the HCFstaff, e.g., the patient procedure for the prevention and treatment ofdecubitus ulcers patient. Unlike some medical procedures which storevideo frames based on, first, the medical procedure being initiated bythe HCF staff, and then opening a predetermined time window associatedwith the procedure for retaining video frames for the subject procedure,the presently described procedure analyzes individual image frames andgroups of frames for content related to the prevention and treatment ofpatient decubitus ulcers, (i.e., . . . decubitus ulcer procedure events)in accordance with the presently described invention. This is necessarybecause, in practice, the medical procedure for the prevention andtreatment of patient decubitus ulcers is ongoing, although only certain“patient turning” events, alert events and timed sample events are ofimportance as documentation or for further review by the HCF staff andretention. These features will become more apparent from the descriptionbelow.

In any case, it should be appreciated that exemplary nonvolatilememories 214 and 215 may be provided in various configurations, such asseparate memory devices or partitions in a single memory. It should alsobe understood that camera control device 210 should have a nonvolatilememory for storing patient medical procedure AN data for future access.Optimally, secondary memory 215 is a nonvolatile memory that is capableof retaining data in the event of a power loss. However, in accordancewith other exemplary embodiments of the present invention, secondarymemory 215 may instead be configured as conventional RAM memory, whereinpriority AN data is copied to a primary nonvolatile memory immediately(such as primary nonvolatile memory 214). While this configurationprovides far less security for the priority surveillance AN data,utilizing smaller RAM memory is usually more economical than utilizing anonvolatile flash memory. Additionally, a portion of secondarynonvolatile memory 215 may be allocated for use by video processor 213.In this configuration, even surveillance data being processed by videoprocessor 213 will be retained in the event of a power failure. Here itshould be mentioned that generally, access time for nonvolatile memorydevices are significantly slower than for comparable volatile memorydevices, hence a tradeoff exists between processing speed andpersistence.

Surveillance system 220, including camera control device 210, along withits ancillary and peripheral components, is connected to a variety ofphysical locations (see FIG. 3), wherein the captured surveillance ANdata may be monitored, retained or otherwise processed for the HCF.Virtually every surveillance system 220 on HCF distribution network 240will be connected to a patient monitoring sub-system for dedicatedmonitoring (depicted in the figure as patient monitoring sub-system 270of nurses station 250). In accordance with one exemplary embodiment ofthe present invention, nurse monitor device 260 of patient monitoringsub-system 270 may be configured identically to camera control device210. In that case, nurse monitor device 260 generally comprisesprocessing unit 261 for storing and executing instructions, networkcontroller 262 for connecting to HCF distribution network 240, videoprocessor 263 and nonvolatile memories 264 and 265 for processing and/ordisplaying any captured surveillance data and/or patient medicalprocedure documentation screens. Video processor 263 may be integratedin a general purpose processing system or supplement the videoprocessing capabilities of the general purpose processing system, whichis coupled to video monitor 267 for viewing. Nurse monitor device 260may also be coupled to a surveillance camera (not shown); in that casevideo processor 263 is also responsible for surveillance video capturedas discussed above with regard to video processor 213. Additionally,patient monitoring sub-system 270 should also comprise audible alarm 268for supplementing visual alarms and warnings displayed on video monitor267. Because patient monitoring sub-system 270 may also be used as asupplemental input device for entering, editing or supplementing theinformation relating to a patient medical procedure, patient monitoringsub-system 270 is typically connected to a variety of data entryperipherals, such as manual interface device 269 (typically a mouseand/or keyboard) and/or touch screen video monitor 267.

FIG. 3 is a diagram an HCF distribution network in which the presentpatient movement monitoring system may be implemented in accordance withan exemplary embodiment of the present invention. HCF distributionnetwork 240 is coupled between patient rooms 230 and various HCFoffices, such as nurses station 250, patient administration offices 345and system administration offices 348 via transmission medium 341 (suchas twisted pair, coaxial conductor, power conductor, optical fiber, airor any other suitable transmission medium). Camera control device 210 islocated in each patient room 230, as well as any other location thatsurveillance and/or monitoring is desired (such as nurse's station 250)and coupled through transmission medium 341 to a variety of networkswitches 342 and other routing hardware. As mentioned, the presentinvention is flexible enough that for many applications, general purposecomputer 310 (i.e., PC, laptop, handheld, palmtop, or other networkdevice) may replace camera control device 310. If broadband access isrequired, HCF distribution network 240 may be connected to broadbandconnection 343 through network server/router/firewall 346. In practice,one or more servers may be connected to HCF distribution network 240,however at least one network storage, HCF network storage 344, should beprovided for maintaining patient and other information, such as patientmedical procedure AN files.

One advantage of the presently described HCF distribution network 240 isthat the network is capable of retaining archival data in adecentralized structure of semi-autonomous storage devices, i.e.,primary nonvolatile memory 214, for instance, in each of patient rooms230. Ideally, all surveillance data from every surveillance camera 217is retained in HCF network storage 344 and all surveillance dataassociated with a patient is retained with that patient's medicalrecords. However, as a practical matter, server storage space andnetwork bandwidth come with a relatively high price. Therefore, the lesssurveillance data transmitted across HCF distribution network 240 andretained in HCF network storage 344, the lower the costs associated withthat data. However, at least the high priority surveillance datadesignated for archival retention and written to primary nonvolatilememory 214 will be available locally in patient movement monitoringsystem 200 at respective patient room 230 for a significant time period,perhaps between several months to in excess of a year. As a practicalmatter, however, lower priority surveillance data may also persist forlong periods of time in patient movement monitoring system 200 dependingon the data handling approach and the amount of surveillance datadesignated as being high priority. During that time period, authorizedHCF staff may access the surveillance data retained at primarynonvolatile memory 214 of patient movement monitoring system 200 in theseparate patient rooms 230 based on a patient identifier, a time period,a procedure or event identifier or the like. In so doing, high prioritysurveillance data of particular interest can be further designated formore permanent retention, such as in HCF network storage 344.Consequently, the decentralized structure of semi-autonomous storagedevices of HCF distribution network 240 enable the HCF to retain asignificant amount of high priority surveillance data for an extendedperiod of time without the additional expense of increased networkstorage capabilities or network bandwidth.

Using the exemplary surveillance, monitoring and network hardwaredescribed immediately above, the workload on the HCF staff can besignificantly and immediately reduced through the implementation of asemiautonomous patient decubitus ulcer procedure by using and analyzingvideo surveillance data in accordance with exemplary embodiments of thepresent invention. Here, the aim is to reduce the overall workload onthe HCF staff professionals, while simultaneously creating a succinctvideo record of decubitus ulcer procedure events by tightly integratingthe patient decubitus ulcers prevention/treatment order with the videomonitoring system.

FIG. 4 is a diagram of a flowchart illustrating a process forimplementing a semiautonomous video patient repositioning procedure inaccordance with an exemplary embodiment of the present invention. Theprocess begins by invoking a patient decubitus ulcer protocol (step402). The patient decubitus ulcer protocol may be derived from a numberof sources, for instance the patient order, the HCF care guidelines,patient repositioning schedules, etc. for documenting a procedure forthe prevention and treatment of decubitus ulcers as a collection ofannotated video frames. Much of this protocol information is manuallyentered, updated or verified in a setup phase of the video monitoringprocedure. While it may be important for the video record of someprocedures to memorialize the procedure as an uninterrupted videosequence, it is far less so with regard to documenting the procedure forpreventing/treating decubitus ulcers. In fact, it is highly advantageousto represent the decubitus ulcer procedure as a chronological group ofindividually time stamped video images, each annotated with relevantprocedure information, that separately represent patient decubitus ulcerevents. Hence, a patient's entire HCF admission, or any sub-portion canbe reviewed as a storyboard of time stamped and annotated video imagesrepresenting decubitus ulcer events.

With the patient decubitus ulcer protocol defined in the system, theprocess enters the run mode in which the system performs two primaryfunctions: it alerts the HCF staff of decubitus ulcer events (step 404),whether the decubitus ulcer events have actually occurred, will occur,are merely potential events or forthcoming events. Additionally, thesystem may also alter the patient herself of a decubitus ulcer eventthat might be resolved solely by the patient, such as a turn alert thatcan be resolved without HCF staff intervention by the patient turningherself into the next scheduled position.

In accordance with still another exemplary embodiment of the presentinvention, patient movement monitoring system 200 uses a set ofescalating alerts and alarms to ensure that the proper attention isbeing paid to the patient's turn requirements. Alerting the patient andthe HCF staff of decubitus ulcer events should not be taken lightly,however some events require far less attention from the HCF staff thanothers. The type of event alert issued by patient movement monitoringsystem 200 is proportional to the seriousness of the event and/or theaction or inaction taken by the HCF staff. For example, standard alertsare issued for events that should be noticed by the HCF staff, buttypically should not require any intervention by the HCF staff. Thealert may take the form of a visual screen message, signal or queue, oran audible tone, beep or verbal message, or a combination of the two.This type of alert is reserved for events of very low importance, suchas the system detecting motion without detecting patient repositioning.It may be resolved by merely acknowledging the alert. The next higherlevel is a warning alert which requires the patient and/or HCF staff totake some type of action within a reasonable time, such as verifying apatient's position or turning a patient and verifying the patient's newposition. A warning alert may also take the form of a visual and/oraudible message, but should be distinctly recognizable from a standardalert. The highest alert level is an alarm alert which requires the HCFstaff to take a specific and immediate action or intervention. Anexample of a decubitus ulcer procedure event that would result in analarm alert being issued if the patient remains in a position for anextended period of time and over the prescribed time for the particularposition. The visual and audible affects associated with an alarm aresharper, louder, more vibrant, frequent and sustained, than other typesof alerts and virtually impossible to ignore.

Although the present alert system is based on discrete alert types,under certain conditions an alert may escalate from one level to thenext higher level if the event is not resolved by the HCF staff. Forexample, if the system throws an event alert such as a patient turnwarning based on an upcoming turn time, it may escalate the alert andassociated alarm if an appropriate HCF staff intervention is notverified to the system. Subsequent to the initial alert and without aresolution of the event, the system repeatedly issues the turn warnings,and may escalate the form of the visual and audible queues to the HCFstaff at each iteration. If the system exceeds some predetermined numberof repeated warnings, the system may automatically escalate the alarm bygenerating an email, text or audible message or HCF electronic postdescribing the alert to an HCF supervisor of the alert event.Additionally, once the seriousness of event exceeds the parameters forthe event warning, the alert level is automatically escalated to thenext appropriate level, for example from a warning event to an alarmevent, and the form of the alert altered for the new level. Here again,the system may automatically escalate the alarm by contacting HCFsupervisor personnel over a predetermined medium.

The system then associates all relevant information attributed to thedecubitus ulcer event with the video image(s) (step 406). With regard toeither alerting the HC professionals of decubitus ulcer events orrecording the event, the video monitoring system uses the patientdecubitus ulcer protocol to discriminate decubitus ulcer events fromnon-events. With that information, video frames that representnon-events are purged from the decubitus ulcer procedure record (step408). Individual video frames and/or groups of video frames aregenerally designated for retention by associating some patient decubitusulcer procedure information with the frame. The information may beretained separately and related to the frame, annotated directly in theframe header or saved in a separate file of related image frames. Framesthat are not associated with some type of patient decubitus ulcerprocedure information are generally not included in the record ofdecubitus ulcer events. As the procedure for the prevention andtreatment of decubitus ulcers will often be implemented simultaneouslywith other patient procedures, it is possible for a single video frameto be associated with two or more patient procedures. Therefore, asingle video frame retained as being representative of two or moreseparate patient procedures and associated with the relevant non-videoinformation for each patient procedure. As a practical matter, thepatient video record may be retained as a single video record or file,that can be parsed into separate patient procedure records by traversingthe record for key information, such as the type of patient procedure.Alternatively, the video patient repositioning procedure may createseparate records of some or all of the active patient procedures thatcan be accessed immediately without the need for analyzing the entirepatient video record.

Two categories of decubitus ulcer events are particularly useful inlessening the workload on the HC professional, alert events andrepositioning events. Alert events are communicated usually to informthe HCF staff of a potential task. The most prevalent of these is theturn alert which alerts the HCF staff that the time interval in aparticular patient position has or will soon elapse. The HCF staffshould then take steps to continue the patient procedure decubitus ulcerprocedure in accordance with the turn schedule and/or patient order.Patient repositioning events involve some information attributable to achange in the patient's position. While this information might bederived from various sources, for instance manually input by the HCFstaff or received from motion detectors, pressure mats, etc., for thepurposes of describing the present invention, the primary source ofinformation attributable to a change in the patient's position is thesurveillance video of the patient. Hence, one or both of patientsurveillance sub-system 220 and patient monitoring sub-system 270, inparticular one of camera control device 210 and nurse monitor device 260analyzes the patient video surveillance for information attributable toa decubitus ulcer procedure event, such as a change in the patient'sposition. While both alert events and repositioning events are recordedas part of the patient decubitus ulcer record, not all patientrepositioning events are communicated to the HCF staff. Some patientrepositioning events, while being a necessary part of the patientrecord, are simply not important enough to alert the HC professional,these include, for instance, sample video frames depicting the patientat certain time intervals.

FIG. 5 is a flowchart depicting a process for implementing a procedurefor prevention and/or treatment of decubitus ulcers using video analysisin accordance with an exemplary embodiment of the present invention.This process is executed at, generally, patient surveillance sub-system220, but might instead be executed in whole or part at patientmonitoring sub-system 270. However, because the processes describedherein are executed on a system network, as a practical matter it ispossible to execute a particular process on almost any network device incommunication with camera 216 in patient room 230. The presentlydescribed method illustrates, generally, a loose relationship betweenthe alert events and patient repositioning events, some of which arederived through analysis of the video frames. The process begins bysetting up the procedure for the prevention and treatment of decubitusulcers, including the decubitus ulcer protocol described immediatelyabove, but also includes setting up the decubitus ulcer monitoringprocess (step 502). With regard to the presently described patientmovement monitoring system 200, most setup procedures will be executedat the interface of patient monitoring sub-system 270. Because thepresent invention will be described with regard to a number of differentanalysis techniques, the set up steps associated with each may vary. Ingeneral, however, one useful preliminary task is to ensure that thevideo camera is aligned directly with the patient area, usually thepatient's bed, and centered thereon. Because the patient area of thevideo is of significant importance to the video analysis, the video viewport axis of video camera 216 should evenly subdivide the patient's bed,longitudinally, into left and right halves. In so doing, the patientwill also tend to lie along the camera view port axis. Additionally, theHCF staff should define a patient area within the video frame usingnurse monitor device 260, which will be discussed further with respectto FIGS. 8-A-8E. In so doing, the amount of video data to be analyzedmight be greatly reduced. Other setup tasks will become apparent withthe discussion of the separate analysis techniques below.

Next, in run mode the process continually tests the current (elapsed)time against a patient repositioning schedule of patient positions andcorresponding time intervals, usually as defined by a patient order(step 504). This test is performed in the background regardless of mostother types of event. At or near the expiration of a time interval or ascheduled event, the system issues an alert event and notifies the HCFstaff of the pending expiration of the time interval as well as the nextscheduled patient position as defined in the order. Typically, thisprocess will run on patient monitoring sub-system 270 for a plurality ofpatients under the care of the charge HCF staff. Once a scheduled turnevent is determined, patient monitoring sub-system 270 simultaneouslyalerts the HCF staff and communicates the event to patient surveillancesub-system 220 which then captures an image frame and annotates theframe for the event. Alternatively, however, the schedule monitoringprocess may run on patient surveillance sub-system 220, which, inresponse to a scheduled turn event captures and annotates an image frameof the patient's current position and simultaneously communicates thealert to patient monitoring sub-system 270, which alerts the HCF staff.

With further regard to the run mode, the process is typically executedon camera control device 210, which continually analyzes the videoframes for changes that indicate one or all of three types of events:motion, patient movement and change in patient position. Essentially,the exemplary process flow detects motion, then attempts to discriminatepatient movement from the motion and finally attempts to detect changesin patient's position from the patient movement. These events aredetected by comparing the current video frame (usually identified as theN^(th) frame) with a previous video frame (usually identified as the(N−k)^(th) frame). Changes in the position of a patient cannot beaccurately assessed unless the patient is at a rest state when theanalysis of the video frame is accomplished, that is all patient motionhas ceased. Any temporal motion and temporal movement will skew theanalysis results. Temporal motion can be detected by testing frame Nwith frame (N−1), if motion is not detected, then no temporal motion ispresent in frame N. Consequently, more accurate video analysis resultsin assessing the patient's position, as well as changes in the patient'sposition, are achieved by using only video frames without temporalmotion, that is, testing frames N and (N−1) for changes in the patientarea that indicate motion prior to assessing patient movement orposition events. If motion is detected between frames N and (N−1), it isgenerally preferable to disregard the current frame N from furtheranalysis and capture a new frame N, and then retest the new video framesNand (N−1) until no motion in the patient area is detected. That frame Nwill be designated as having no temporal motion (NTM) for furtherreference. Again, for most accurate analysis results, most techniquesare better served using video frames without temporal motion {NTMframes). Hence, the video analysis detects motion between video frames(step 506) for two possible purposes: to identify NTM frames (whereframe N=frame (N−1) in the patient area); and to identify video framesfor further analysis to delineate patient movement in the area betweenframe N and the previously saved NTM frame (the previously saved NTMcannot be the previous video frame (i.e., frame N−1)).

Next, the automated process attempts to discriminate patient movementsfrom motion. Patient movement can never be detected in a video framewithout any motion. However, not all motion in the video frame can beattributable to patient movements. Therefore, a further aim of thepresent invention is to discriminate any motion from consideration thatcould not be attributed to a patient's movement, such as motion awayfrom the patient. Patients will move their heads, arms, legs and evenmove their torsos, any of which may or may not be an indication of apatient repositioning event. Therefore, the motion is tested using oneor more of the techniques describe below to decipher patient movementfrom motion in the current video frame (step 508). For instance, patientmovement may be deciphered if a part of the patient changes positionfrom the previous NTM video frame. Depending on how the decubitus ulcerprocedure is configured, the HCF staff may be alerted to the patient'smovement. However, what is actually sought is to detect changes in thepatient's position from patient movement (step 510). When a patientchanges her position from the patient order, the HCF staff should actimmediately to reposition the patient in accordance with the order. Onetechnique is to infer a change in position using frame analysis based onthe amount of patient movement; if the level of movement exceeds athreshold amount, a change in position is inferred and the HCF staffalerted (discussed below with regard to FIGS. 9A, 9B 10A and 108). Othertechniques attempt to identify the patient's position, i.e., back, rightside, left side, etc., and compare that position with the patient'sposition in the previous NTM frame (discussed below with regard to FIGS.11A, 118, 12A-12F, 13A-13L, 14A-14C, 17, 18A-18E, 19A, 19B, 20A, 20B, 21and 22). If the two positions are different, the HCF staff is alerted.The latter technique may require an immense amount of analysis toidentify a particular patient position. In any case, once the patient'scurrent position is identified, it is compared to the patient order(step 512). Where a conflict with the order is detected (i.e., the wrongpatient position for the time), the HCF staff is alerted with allrelevant information (such as the current patient position, the correctpatient position by the order, time remaining in the position andperhaps the next patient position according to the order). Obviously,there may be cases where the patient has repositioned himself contraryto the order, but to an acceptable alternative position. One such caseinvolves the treatment of decubitus ulcers at a particular site, forinstance the patient's back. The attending physician may order that thepatient be kept off her back for an hour of each hour and a half. There,the patient may be free to move from one side to the other as much asnecessary to keep comfortable so long as the patient does not stay onone side more than a half hour or return to her back. In thosesituations the process would detect changes in the patient's position,perhaps alert the HCF staff to the changes, but recognize that thechanges do not conflict with the patient order.

The process described above in FIG. 5 is an extremely high revel view ofthe present process for procedure for prevention and/or treatment ofdecubitus ulcers using video analysis in accordance with an exemplaryembodiment of the present invention. FIGS. 6A and 6B depict a flowchartof a lower level description of the generic process, in accordance withexemplary embodiments of the present invention. Here, the process isdescribed at the level of an individual video frame. However, beforedescribing the generic process a brief description of video capture datamight be helpful. Generally, a typical video camera captures a sequenceof digital video frames, each at a predetermined resolution, at apredetermined frame rate, for instance sixteen, twenty, twenty-four andeven up to thirty-six frames per second. The resolution of digitalcameras is usually defined by the number of pixels both horizontally andvertically (such as 640×480) or as a total number of pixels in the image(such as 1.4 mega pixels), while the resolution of analog video camerasis typically defined by the number of television lines, for instance 320lines: Most prior art digital video cameras used for surveillanceapplications have a relatively low resolution, in the range of 640×480pixels, however the resolution is often dependent on the application andthe size of the surveillance area in the field of view of the camera.For instance, a larger surveillance area might necessitate the use of ahigher resolution camera in order to recognize objects in the field ofview. As patient rooms in a HCF are relatively small, adequatesurveillance video has been achieved using relatively low resolutioncameras. Maintaining resolutions as low as possible is often importantbecause the AN traffic on network 240 often creates bottlenecks forother, sometimes more important, network traffic data (see generallyU.S. Pat. No. 7,477,825 discussing techniques for handling AN networktraffic). Hence, video resolution often suffers for the sake of networkflow.

Furthermore, for the purposes of describing the video processing andanalysis techniques below, it should be appreciated that each pixel ofthe 1.4 mega pixel exemplary frame alluded to above, is defined by twooptical characteristics: chrominance and luminance. Using luminance andchrominance image encoding, each pixel of a frame is allocated apredetermined number of bits for describing the pixel, for instance a 12bit pixel encoding in the YUV12 planar pixel encoding devotes 8 bits tothe Y, or luminance channel, and the remaining 4 bits allocated for thechrominance of a 2×2 pixel block. In any case, the chrominance bitsdescribe the pixel's color (or the coordinates of the color in somepredetermined color space), while the luminance bits describe theintensity of the light. Hence, chrominance pixel characteristics may beextremely useful in recognizing a patient in a video frame by skin tone,or other bodily colorations. However, the use of chrominance pixelcharacteristics is susceptible to changes in the ambient brightness ofthe surveillance area, as the ambient light in the area diminishes, thechrominance bits fall within a narrow range of color coordinates in thecolor gambit, typically near the gray scale. While it may still bepossible to distinguish the patient from the surrounding area usingcontrasts in the chrominance characteristics of the pixels in the videoframe, the process is far more difficult and less reliable.

Luminance pixel characteristics, on the other hand, are far lesssusceptible to changes in the ambient brightness of the surveillancearea for two reasons. First, the amount of light reflected off a surface(the luminance value) is highly influenced by the distance from thelight source in the surveillance area and that distance can be generallyreferenced to a patient's position in a surveillance area. Assumingillumination in a patient's room is provided by one or both of anoverhead room light and a video camera equipped with infraredillumination elements that are generally arranged around the lens andoriented toward the view axis of the camera (i.e., coaxial to thecamera's view axis), optimally the patient's bed should be positioned atthe cross-section between the two illumination vectors. That is, thepatient's bed should be positioned directly under and centered with theoverhead room light and directly in front of and centered on the IRillumination from the camera. In so doing, the luminance values alongany lateral stripe along the view port, and hence a video frame, willvary proportionally with the relative distance to the light source,either the overhead light of their elements on the camera. In a smallpredefined area of the view port, for instance the patient's bed, thevariance of the luminance may be negligible. Hence, changes in theluminance across lateral stripes of the patient's bed will be highlyindicative of changes in the relative height of the target surface onthe patient's bed. These changes in height are largely due to thepatient's position on the bed. For simplicity, the directionalnomenclature used hereinafter will correspond to the image frame, forinstance since, optimally, the patient's bed is aligned along the viewport axis of the camera, vertical runs from along the length of thepatient's bed (from top (headboard) to bottom (footboard)) (verticalpixel lines in an image frame) and horizontal runs across the bed(horizontal pixel rows in an image frame), that is, horizontal traversesthe lateral extent of the bed. Consequently, it is sometimesadvantageous to process pixels in a video frame along horizontal pixelrows when attempting to identify the pixels corresponding to therelative highest elevated locations along the stripe in the surveillancearea, because pixels of like elevation will have similar luminancevalues.

While evaluating the luminance characteristics has some advantages overusing chrominance characteristics, and vice versa, the procedure forprevention and/or treatment of decubitus ulcers using generic videoanalysis is not necessarily dependent on the luminance characteristics,chrominance characteristics or some other pixel characteristic. Hence,luminance and chrominance will be generally referred to throughout ascolor temperature of a pixel, group of pixels, area of the image frame,etc.

Turning now to FIGS. 6A and 6B, a lower level view process forimplementing a procedure for prevention and/or treatment of decubitusulcers using generic video analysis is depicted in which alert eventsand decubitus ulcer procedure events are illustrated with regard to acurrent image frame in accordance with an exemplary embodiment of thepresent invention. While it is contemplated that this process is largelyperformed with camera control device 210, as a practical matter theprocess might also be executed within patient surveillance sub-system220 in nurses station 250 or any other location within the HCF, such asadministration offices 345 and/or system administration offices 348. Theprocess continues in an iterative loop during the patient's stay at theHCF, or at least while the patient order is active, each cycle beginningwith the capture of a digital image frame, current frame N (ordigitizing of a captured analog frame) (step 602). Initially, thecurrent elapsed time with the patient in the current patient position iscompared to the patient order, the turn schedule or some other metric,in order to determine if it is time to turn (reposition) the patient(step 604). If the process determines that it is not time to turn thepatient, it flows directly to step 608 without executing step 606. If itis time, nurse monitor device 260 of patient monitoring sub-system 270issues a turn alert, such as a visual warning displayed on video monitor267 and/or audible alarm from audible alarm 268. Alternatively, thesystem may issue the alert to the patent herself, thereby relieving theHCF staff of alerting the patient. For example, patient movementmonitoring system 200 issues an audible turn alert via medicalprocedure/pillow speaker interface 226 with the turn information, forinstance, “The time is now 12:36 PM. You must turn from your back toyour left side. If you are having difficulty or do not understand theseinstruction, please push the Nurse Call Button on your remote.”

The event alert is then recorded by, for instance, annotating currentimage frame N with the decubitus ulcer procedure event information and atimestamp or by associating the event data stored at a separate locationto the current image frame. The event information typically includes thealert event, the current patient position, elapsed time in thatposition, the next position, verification information from the HCF staffand any other information that may be useful in evaluating the procedurefor prevention and/or treatment of decubitus ulcers. Next, optionally,the process identifies an area of interest in frame N that shouldcontain the patient, as discussed with regard to step 502 depicted inFIG. 5 (step 608) and as will be further discussed with regard to FIGS.7 and 8A-8E below. Providing a description of the area of interestreduces the amount of image frame to be analyzed to only the portion ofthe image frame where the patient is known to be present, therebyaccelerating the analysis procedure. Without the description of the areaof interest, one of camera control device 210 and patient surveillancesub-system 220 will evaluate the entire image frame in the mannerdiscussed below. With the area identified, frame N is tested fortemporal motion, that is, if the pixels defining the area of interest inframe N are substantially unchanged from the pixels defining the area ofinterest in the previous image frame, frame (N−1). It should beappreciated that patient repositioning takes from several to hundreds ofimage frames to complete and each of the transitional image framesmerely exhibit temporal motion without necessarily showing any usefulpatient position information. Therefore, if frames N and (N−1) are notsubstantially similar, at least in the area of interest, frame N isflagged as having temporal motion (step 612). Image frames with temporalmotion are generally discarded sometime after frame (N+1) is receivedand, therefore, flagged for deletion at cleanup (step 614), in favor offrames showing the patient at rest without motion. Here it should beappreciated that in most image analysis embodiments discussed herein,each image frame N is tested for temporal motion by comparing thecurrent frame to the preceding image frame (N−1) for motion exceeding apredetermined threshold amount, at least in the area of interest. ThisNTM test is performed regardless of whether or not frame (N−1) has beendesignated as an NTM frame, hence the preceding image frame (N−1) isgenerally saved in secondary nonvolatile memory 215 at least until theNTM test for the next frame N is completed.

Typically, temporal image frames are cached temporarily in smallersecondary nonvolatile memory 215 and removed only when space in neededfor more current image frames. Image frames residing in secondarynonvolatile memory 215 are only written to larger primary nonvolatilememory 214 if the frame is annotated with decubitus ulcer eventinformation and/or if the frame is annotated to be saved at cleanup. Inso doing, image frames without useful decubitus ulcer procedure eventinformation are deleted from secondary nonvolatile memory 215 wheneverthe memory is cleaned to make room for more current image frames. Asdiscussed in U.S. Pat. No. 7,477,825 and elsewhere in application Ser.Nos. 12/151,242 and 12/589,654, image frames often reside in secondarynonvolatile memory 215 while waiting to be transmitted over network 240to patient monitoring sub-system 270. Due to a myriad of reasons, someof these frames may never reach nurse monitor device 260 and thereforebe copied to primary nonvolatile memory 214 for future reference. Also,although the scope of this application is limited generally toprocedures for prevention and/or treatment of decubitus ulcers usingvideo analysis, other patient procedures will often be runningsimultaneously, hence frame N might be designated for deletion bydecubitus ulcer procedure, but designated for saving for another reason,such as by another patient procedure.

Returning now to step 614, the process reverts to step 602 for anothervideo frame. However, if at step 610, frame N is determined to have notemporal motion, the frame is flagged as a NTM frame (step 616) and thecurrent NTM frame N is compared to the most recent NTM saved in memoryfor patient movement (step 618). That is, using some image analysistechnique, at least the area of interest in frame N that contains thepatient is analyzed for patient movement when compared to the last NTMimage frame. If the results of the image analysis indicate that themotion detected in NTM frame N is not patient movement, then frame N hasno additional decubitus ulcer procedure event information over thepreceding NTM frame and therefore frame N can be discarded, in favor ofsaving the older NTM frame (step 614). Alternatively, since NTM frame Nand the preceding NTM frame convey identical information concerning thepatient's position, either frame may be designated as representative ofthe current patient position state or decubitus ulcer event.

Returning to step 618, if patient movement is detected in frame N, frameN should be analyzed further to determine if that movement reached thelevel of a repositioning. One mechanism is to assess if it is possibleto determine or identify the patient's current position in frame N (step620). Depending on the analysis technique being used, or other factors,the patient's current position may not be recognizable using theanalysis techniques being employed. If that is the case, a change in thepatient's position might be inferred from other factors, such as theamount of patient movement detected even if the current patient positionremains unknown (step 622). Additionally, the inference of patientrepositioning might also be substantiated by detecting movement outsidethe area of interest where the patient resides in the frame immediatelypreceding and possibly subsequent to the NTM frame exhibiting patientmovement. It is a reasonable inference that the HCF staff entered thepatient's room and repositioned that patient, even though it may not bepossible to determine the patient's current position using the imageanalysis technique currently being employed.

Returning to step 622, if the system detects patient movement, but thepatient's current position cannot be determined nor can a change in thepatient's position be inferred, it is still possible that the patienthas repositioned herself, hence, the HCF staff is alerted to thepossibility of a patient repositioning event (step 626), a request forHCF staff verification issued (step 628) and the current NTM frame Nannotated with the decubitus ulcer event, timestamp and designated tosave at cleanup (step 630). From here, the process iterates again tostep 602 for the next video frame N. Returning to step 622, if theresults of the current analysis technique infers that the patient hasrepositioned herself, a repositioning event has occurred and, as aresult, patient movement monitoring system 200 activates a patientrepositioning alert at nurse monitor device 260 to alert the HCF staffof a potential improper repositioning event, that is one that is notscheduled, to a position that conflicts with the patient order, or both(step 632).

The process loop across step 622 assumes that the patient's position isundeterminable, however, if at step 620, the patient's position can bedetermined, than that patient position is compared to the last NTMframe. If the potentially new patient position is not a change from thepatient's position in the last NTM frame, a request for HCF staffverification of the nonevent may still be issued as a safeguard (step628) and the current NTM frame N annotated with the decubitus ulcerevent, timestamp and designated to save at cleanup (step 630).Periodically, a patient may adjust her position without actuallychanging positions, and patient movement associated with theseadjustments might generate decubitus ulcer events that may bemisinterpreted by the automated system process, therefore, the systemseeks verification of the patient's positional state from the HCF staff.

Alternatively, if the patient movement is attributable to arepositioning event (step 624) a patient repositioning alert isactivated at nurse monitor device 260 and the new patient position isautomatically compared to the patient order for validity (step 634). Theaim here is to identify patient repositioning events that correlateclose enough with the patient order (or turn schedule or HCF decubitusulcer procedure guidelines) to be compliant, without generatingunnecessary HCF staff intervention. Therefore, no additional attentionis necessary from the HCF staff, aside from verifying the patient'sposition (step 636), resetting the repositioning alert and timer (step638) and frame N with the annotating decubitus ulcer event, the HCFstaff verification, timestamp and to designating the frame for saving(step 640). It should be mentioned that it is not possible for the HCFstaff to verify the alert, patient position or anything concerning thecurrent decubitus ulcer event simultaneously with frame N beinganalyzed. Hence, steps 366, 638 and 640 actually occur subsequent to thevideo analysis of frame N, although the frame itself may be availablefor annotating in one or both of secondary nonvolatile memory 215 andprimary nonvolatile memory 214. The process then reverts to step 602 foranother video frame. If, however, at step 634, if the new patientposition does not conform to the patient order, then the HCF staffshould act to rectify the patient's position to one that conforms to theturn schedule. There, the HCF staff is alerted to the possibility of anoncompliant patient repositioning event (step 626), a request for HCFstaff verification issued (step 628) and the current NTM frame Nannotated with the decubitus ulcer event, timestamp and designated tosave at cleanup (step 630). The process then reverts to step 602.

Returning to the high level process flow depicted in FIG. 5, thepresently described invention may be subdivided into several discretesubparts: setup phase (step 502); monitoring phase (step 504); imageanalysis phase for detecting and evaluating patient repositioning events(steps 506, 508 and 510) and a compliance phase where the results of theimage analysis phase are compared to the patient order (step 512).Consequently, the remainder of the specification is subdividedsimilarly: FIGS. 7 and 8 depict tasks accomplished in setup phase; FIGS.14 and 16 depict monitoring phase events and screenshots; while FIGS. 9,10, 11, 12, 13, 15 and 17 through 22 depict various analysis techniquesapplicable to the image analysis phase for deriving useful repositioningevent information from the image frames. Finally, although notillustrated in FIG. 5, FIGS. 23 and 24 illustrate processes for creatingand compiling a historical record of patient decubitus ulcer procedureevents from the annotated image frames maintained in storage.

Before discussing the various analysis techniques of the presentinvention, it is important to remember that currently there is nointegrated system for using a video monitoring system to prevent andmanage decubitus ulcers for a group of patients. Therefore, as athreshold the present invention enables attending HCF staffprofessionals, who are monitoring the video images of the patients onpatient monitoring sub-system 270, to manually record patient movement,restarting the timer after each HCF staff intervention. Essentially, oneof patient monitoring sub-system 270 and patient surveillance sub-system220 tracks patient turn intervals (based on a turn schedule compliantwith a patient order and/or HCF care guidelines for treating/preventiondecubitus ulcers) and patient monitoring sub-system 270 alert the HCFstaff that it is time to change the patient's position. The HCF staffthen repositions the patient. Finally, the HCF staff uses the touchscreen interface 267 to select the patient, and indicates that thepatient was repositioned manually. The system then automaticallydesignates one or several current image frames to be saved as a positionrecord for the patient. Alternatively, the system automaticallydesignates an image frame of the patient for the patient positionhistory simultaneously with notifying the HCF staff to change thepatient's position. This frame is that pre-event frame and the former isa post-event frame.

However, it is important to reduce as much manual intervention from theHCF staff as possible. Below are described several analysis techniquesfor accurately tracking a patient's position, with respect to a turnschedule, for instance, which may be used, for example, to automaticallyalert the HCF staff of incompliant patient positions and thereby reduceHCF staff monitoring efforts and manual interventions with the patient.The processes typically begin by setting up the patient and system ofthe particular technique.

FIG. 7 is a screenshot of a setup page for a procedure for preventionand/or treatment of decubitus ulcers in accordance with an exemplaryembodiment of the present invention. Setup includes at least threediscreet manually implemented tasks: setting up the patient room tooptimize the automated position/movement/change detection analysis;setting up the video image to optimize the automatedposition/movement/change detection analysis; and setting up patientorder and procedure execution information. The screenshot in FIG. 7depicts the latter of the three. This setup may be implemented similarlyto that described in U.S. application Ser. No. 12/589,654 directed tothe ProcedureView product available from CareView Communication, Inc. ofLewisville, Tex. and described elsewhere above. Essentially, screenshot700 contains patient information 740, Date/Time/Room information 720,and procedure information 730. The HCF staff begins by bringing up thepatient information by typing the patient's name in text entry 740, orselecting the room number for the patient in entry 720. In either case,screenshot 700 is populated with patient information that is thenverified by the HCF staff before entering new patient procedureinformation. The present setup procedure is relatively uncomplicatedwith auto-fill features wherever appropriate. For instance, once thepatient's screen has been displayed, the HCF staff merely selects thatcorrect procedure in entry 730, or alternatively the procedure billingcode. In response, the comment entries, warning, alert and samplingselection, as well as the Turn Schedule selection box are automaticallydisplayed for HCF staff editing. Once displayed, the HCF staff enterspatient comments respective of the patient order, such as a descriptionof the ailment “stage II decubitus ulcers present on right hip andankles with stage III ulcers on the patient's right shoulder.” The HCFstaff then enters procedure events, for instance by merely selectingposition options corresponding to the patient order. In accordance withthe exemplary setup screen, a plurality of patient position selectionsand corresponding timer selections are displayed in turn schedule 710.As depicted in the example, the patient order might specify 30 minuteson the Back position, followed by 40 minutes on the Left Side position,then 20 minutes on the Rights Side position, followed again by 40minutes on the Left Side position. The system calculates the positioncycle summary 714 including the number of position changes per cycle andthe cumulative cycle time. The HCF staff can readily appreciate the needfor limiting the patient's time on back and right side due to theeruption of ulcers on those parts. Additionally, the HCF staff selectsthe types of automated detections to be performed, such as patientposition selection 702, as well as selecting position change alarms 704to alert the HCF staff whenever the system detects a change in thepatient's position. The HCF staff may also designate other warnings andalarms, such as issuing patient movement warnings 706 when the systemdetects movement. Additionally, the system may save sample framesdepicting the patient at certain time intervals set by the HCF staff inpatient turn schedule 710. Finally, the HCF staff also sets the timeinterval for a historical position report using rolling time intervalselection 709. A historical position report for the patient is thendisplayed for the time interval selection at the click of an icon. Forexample, when queried, the system will display all repositioning andalert events and perhaps sample position frames, for the immediatelypreceding 90 minutes.

FIGS. 8A-8E illustrate another setup task that greatly optimizes theexecution of the present invention, that of describing an area ofinterest within the video frame where the patient resides. Onceaccomplished, most image analysis techniques will be confined to thatarea. Typically, the area of interest is the patient's bed, but may beany area where the patient will reside for periods of time, e.g.,gurneys, chairs, wheelchairs, treatment apparatus, etc. The aim ismerely to place a bounding box around the patient area in the view areaof screen 267. As depicted, bed 321 may not be completely parallel withthe view axis, which may occur in physical locations where camera's 216position is not directly in line with patient bed 321. This setup may beimplemented similarly to that described in U.S. application Ser. No.12/151,242, directed to the Virtual BedRail product available fromCareView Communication, Inc. The HCF staff begins by tapping touchscreen 267 or double clicking pointer 832 to bring up bedrails 840.Bedrails 840 are aligned on the upper extent of bed 321 (FIG. 8A),extended downward over the lower extent of bed 321 (FIG. 88), skewedhorizontally over the surface of bed 321 (FIG. 8C) and finally skewedvertically over the remaining surface of bed 321 (FIG. 80). Virtualbedrail 840 is then superimposed over the area of interest in atransparent or unobtrusive color that does not inhibit the view ofpatient bed 321. Alternatively, in a more automated procedure theoperator sequentially taps on the view area of screen 267 at the cornerpositions of the intended area of interest, such as at the corners ofpatient bed 321 or around a patient's gurney, chair or wheelchair. Thesystem then automatically generates bedrails 840 on display screen 267.

FIGS. 9A, 9B, 10A and 108 illustrates one exemplary method forautomatically assessing changes in the position of a patient using videoin accordance with one exemplary embodiment of the present invention.The present example depicts the patient in a recumbent position onpatient bed 321 for simplicity and is not intended to limit the scope ofthe present invention. The patient may be, in addition to recumbent in abed, seated in a chair or wheelchair or in some other position at, forexample, a medical diagnostic or therapy apparatus. FIGS. 9A and 9Bdepict a flowchart showing a method for assessing changes in theposition of a patient using video by evaluating discrete zones and FIGS.10A and 10B depict a table of image analysis results that may be held insystem memory and a corresponding diagram of the zones (therepresentation of the zones is presented merely to illustrate theprocess and need not be rendered for the image analysis). Essentially,this method subdivides that area of interest within the virtual bedrailsinto n equal-sized zones (i=1 to n), determines which zones the patientis present and then monitors those zones for changes that indicate thepatient's position has changed. In so doing, the system need only tracka subset of all image analysis information and, therefore, need notreanalyze the entire area of interest or recalculate all of the values.The process begins by receiving a definition of the area of interestthat might be created as discussed above with regard to FIGS. 8A-8E(step 902). Next, the process determines vertical axis 1010, horizontalaxis 1012 and subdivides the area of interest 840 into n equally-sizedzones. For example, a camera resolution of 640×512 (327,680 pixels) andan area of interest of 256×300 (81,920 pixels), each zone having 4×4(16) pixels or 64×80 zones, where n=5120, with 2560 zones belowhorizontal axis 1012 and 2560 zones above horizontal axis 1012.

The process begins by receiving a video frame (step 906) and thenscanning the horizontal pixel rows by reading values for each pixel ineach row (step 908), with reference to the diagrams in FIGS. 10A and108, each of the 256 pixels of the 160 horizontal pixel rows in the areaof interest are scanned in this fashion. The pixels may be scanned leftto right or right to left and the pixel values may be either chrominanceor luminance. Next, each pixel is assigned a color temperature based onthe scanned value (step 910). In accordance to one exemplary embodiment,the color temperature equates to the scanned value, however, accordingto another exemplary embodiment each color temperature value correspondsto a range of scanned values so that the amount of discrete values beingsaved is reduced. Next, boundaries of like color temperatures aredefined and like colors are grouped into discrete areas (step 912).

One exemplary mechanism for determining boundary conditions and groupinglike color temperatures is by using high contrasts to delineate pixelboundaries. A boundary along a horizontally scanned row can bedetermined from a contrast threshold between adjacent pixels in the row.For example, scanning from left to right, any pixel that exceeds acontrast threshold with its neighbor to the left is considered acandidate boundary pixel. It is possible for several sequentialcandidate boundary pixels to exceed the contrast threshold. If this isthe case, any of the candidate pixels may be selected as the boundarypixel, however the color temperature of the pixel furthest to the rightshould be most like the color temperature of the pixels filling theinterior of the boundary area. Additionally, along any horizontal pixelline where a first boundary is detected that defines the left side ofthe patient, a second boundary should also exist that defines the rightside of the patient. It is assumed that the boundary condition acrossall pixel rows will define an essentially continuous outline of thepatient's position in the area of interest. Thus, if the contrastthreshold is exceeded by scanning from the left, then further along thepixel row to the right, a negative contrast threshold should also becrossed that designates the boundary condition on the right side of thepatient. The candidate pixel selection rules are essentially identicalfor the right side as the left. The absolute color temperature valuesfor the boundary is relatively unimportant, what is being sought is aboundary that is essentially filled with a like color temperature valuethat can be correlated as the patient. Typically, the largest area oflike color temperature will be the patient and that color temperature isnoted as the patient color.

Next, a weight (W(i)_(N)) for each of the n zones in the area ofinterest of frame N is calculated and saved. Weight (W(i)_(N))correlates the number of pixels in a zone having the patient colortemperature (step 912), i.e., the weight (W(i)_(N)) of any zone in frameN is between 0 and 16, according to the above example and as depicted inthe third column of the tables in FIGS. 10A and 108. Next, thecumulative zone weight (ΣW(i)_(N) ^((1−n))) of all zones n for frame Nis summed, the cumulative zone weight of the zones in the top section(zones 1−j) is summed (ΣW(i_(TOP))_(N·j) ^((1−j))) and the cumulativezone weight of the zones in the bottom section (zones j+1)−n) is summed(ΣW(i_(BOT))_(N) ^(((j+1)−n)))). The cumulative weight of the n zones inframe N is compared to the cumulative zone weight of the same n zones ofthe previous frame (ΣW(i)(_(N−1)) ^((1−n))). The absolute value of thedifference between the two cumulative values (Δ|ΣW(i)(_(N−(N−1)))|) iscalculated, as is the absolute cumulative zone weight differencesbetween the two top sections (Δ|ΣW(i_(TOP))(_(N−(N−1))) ^((1−j))|) andthe absolute cumulative zone weight differences between two bottomsections (Δ|ΣW(i_(BOT))(_(N−(N−1)))|^(((J+1−n))). The zone weights(W(i)_(N), the cumulative zone weight (ΣW(i)_(N) ^((1−n))), the absolutechange in cumulative zone weight from the previous frame(Δ|ΣW(i_(TOP))(_(N−(N−1))) ^((1−j))), the absolute change in cumulativezone weight of the top section from the previous frame(Δ|ΣW(i_(TOP))(_(N−(N−1))) ^((1−j))) and the absolute change incumulative zone weight of the bottom section from the previous frame(Δ|ΣW(i_(BOT))(_(N−(N−1))) ^(((j+1−n))|) are all saved.

Next, temporal motion in frame N is tested by comparing a respectivethreshold value to one or all of the absolute change in cumulative zoneweight from the previous frame (Δ|ΣW(i)_((N−(N−1)))|), the absolutechange in cumulative weight of the top section from the previous frame(Δ|ΣW(i_(TOP))_((N−(N−1))) ^(((1−j))|) and the absolute change incumulative weight of the bottom section from the previous frame(Δ|ΣW(i_(BOT))(_(N−(N−1))) ^(((j+1−n))|) (step 918). If temporal motionis detected, then the values for frame N are saved temporarily forcomparison with the next image frame for temporal motion, and theprocess reverts to step 906) for another image frame. If, on the otherhand, no temporal motion is detected, then current frame N is flagged asa no temporal motion frame (NTM) (step 922), which can then be comparedto a previous NTM frame in order to determine if the patient has changedposition.

Next, zone weights W(i)_(N) and W(i)_(NTM), cumulative zone weightsW_(N) and W_(NTM) are retrieved from memory. Changes in zones' weightsfor zones 1 to n is calculated between NTM frame N and the previouslysaved NTM frame (ΣΔW(i)_((N−(NTM))) ^((1−n))), as is the sum of theabsolute change in the weight of zones 1 to n from the previous NTMframe to frame N (Σ|ΔW(i)_(N−(NTM))|), in only the zones in the topsection (Σ|ΔW(i_(TOP))(_(N−(NTM))) ^((1−j))|) and in only the zones inthe bottom section weight (Σ|ΔW(i_(BOT))_(N−(NTM))) ^(((j+1)−n))|) (step924). ΣW(i)_((N−(NTM))) ^((1−n)), (Σ|ΔW(i)_(N−(NTM))|),(Σ|ΔW(i_(TOP))_(N−(NTM)) ^((1−j))|) and (Σ|ΔW(i_(BOT))_(N−(NTM))^(((j+1)−n))|) are all saved (see the tables in FIGS. 10A and 108).

Finally, changes in the patient's position can be assessed by comparinga respective weight threshold to one or all the absolute cumulativechanges between the current NTM frame N and the last saved NTM frame, iall n zones (ΣW(i)_((N+(NTM))) ^((1−n))), or only the zones in the topsection (Σ|ΔW(i_(TOP))(_(N−(NTM)) ^((1−j))|) or only the zones in thebottom section weight (Σ|ΔW(i_(BOT))_(N−(NTM)) ^(((j+1)−n))|) (step926). If the threshold is exceeded, then a patient repositioning eventhas occurred and the process can invoke a repositioning eventsub-routine, such as exemplary steps 632-640 and 626-630 discussed withreference to FIGS. 6A and 6B.

The preceding process might be optimized by making several assumptions.First, it might be assumed that the patient's color temperature isgenerally constant over long time periods, if so steps 908 and 910 canbe executed periodically instead of every frame. Additionally, if it isassuming the higher weighted zones that represent the patient are highlyindicative of a patient repositioning event, then step 912 might also beomitted until and unless a repositioning event is detected at step 914,then fresh zone weights W(i)_(N) and W(i)_(NTM) should be recalculatedby iterating thought the entire process. Similarly, if zonesrepresenting certain parts of the patient body can be identified thatshould change positions wherever the patient repositions herself, suchas shoulders, hips or face, then only those zones need be analyzed forpatient repositioning events.

The preceding process evaluates patient repositioning as the totality ofpatient movement within the area of interest. While this method isextremely reliable, it cannot identify a patient's position, nor can itdistinguish one position from another. It is therefore advantageous touse feature analysis of the image frame to predict the patient'sposition and orientation. One exemplary method involves detecting theorientation of the patient's head from the image frame. When the patientis on her back, the patient's head is facing forward with both eyes andears in view. When the patient is on her right side, the patient's faceis to the left in the image frame (the patient's right), with only herleft eye exposed near the left boundary of her head in the image. Theleft ear may also be visible, but near the center of the head. Finally,when the patient is on her left side, the patient's face is to the rightin the image frame (the patient's left), with only her right eye exposednear the right boundary of her head in the image, with the right earcentered. Thus, the patient positions of recumbent on her back may bedistinguished from recumbent on her left side and further distinguishedfrom recumbent on her right side. Thus, this analysis approach has thefurther advantage of only limiting the more comprehensive analysis tothe patient's head area.

Thus, the presently described system and method for using a videomonitoring system to prevent and manage decubitus ulcers in patientswill record an image of the patient, and will detect the location of thepatient's face relative to the rest of the body. Initially, the area ofinterest, patient bed 321 is identified manually by the HCF staff as arectangle defined by the four bounding corners. The patient will beidentified by finding pixels which contrast with the actual bed. Thepatient's face will again be identified by finding pixels which contrastwith the remainder of the patient's body. The position of the facerelative to the remainder of the head will be used as the directionalorientation of the face, which will be recorded and monitored over time.If the face is reoriented to face a different direction, the patientwill be recorded as having moved. This process has a major advantageover other analysis techniques where patient's orientation is difficultto track visually due to the fact that the majority of the body iscovered by a sheet. Other advantages will become apparent with thedescription of the process with regard to the figure drawings.

FIGS. 11A and 11B area flowchart depicting a process for assessing apatient position by detecting the position and/or orientation of facialfeatures in a video frame in accordance with an exemplary embodiment ofthe present invention. The process begins by receiving setup informationsuch as the description of the patient area in the frame, similar tothat discussed elsewhere above (step 1102) and receiving frame N (step1104). The remainder of the process comprise two discrete parts:distinguishing the patient's head; and distinguishing patient featureson the head. Distinguishing the patient's head is similar to the processdiscussed with regard to FIGS. 9A and 9B, that is by scanning horizontalpixel rows, reading each pixel value (chrominance or luminance) (step1106) and deriving the boundary between areas of like colortemperatures, the largest continuous area will be the patient (step1108). This area should define the general shape of the patient. Again,for this process, only the portion of the frame showing the patient'shead need be analyzed further, represented diagrammatically in FIGS.12A-12F. FIGS. 12A-12C illustrate the process with the patient on herback and FIGS. 120-12F illustrate the process with the patient on herleft side (right side omitted but is the mirror image of the left side).

Beginning at the uppermost area identified as patient 1202 in the image,scan down toward the patient's feet noting the widths and identify thepatient's head and shoulders and create shoulder line 1210 between thelateral extents of the shoulders (step 1110). The portion of the imagebetween the uppermost area identified as patient 1202 and shoulder line1210 will contain one or both eyes (1204L and 1204R) and one or bothears (1206L and 1206R).

Here, frame N should be tested for temporal motion with frame (N−1)(step 1112). If present, the process should revert to step 1104 for anew frame N. Assuming no temporal motion in frame N, scan the patientface, within box 1214, for areas of high contrast to identify one orboth of eyes 1204L, 1204R and ears 1206L, 1206R (step 1114) (see FIGS.12C and 12F).

Next, the average lateral distance between the top of the patient andthe shoulder line is calculated (represented in the diagrams as box1214) and then divide that area into right and left halves representedin the diagrams as center line 1212) (step 1116) (see FIGS. 128 and12E). Determine the patient's position from the visibility and positionsof eyes 1204L, 1204R and ears 1206L, 1206R (step 1118) as follows: oneeye visible on right facial side with one ear visible in proximatemiddle or across both sides—patient on left side; one eye visible onleft right facial side with one ear visible in proximate middle oracross both sides—patient on right side; and both eyes visible on rightand left facial sides, ears, if visible on right and left facialsides—patient on back.

With the current patient's position being identified, the process canassess changes in the patient's position by comparing the position ofthe patient in current frame N with the position of the patient in theprevious NTM frame (step 1120). The process then reverts to step 1104for the next frame N.

The analysis process above can correctly identify the position of thepatient only if the facial features are identifiable. In some cases thecontrast of the patient's ears or eyes in the image frame is notsufficient even though larger features like the patient's head isclearly distinguishable. This is often caused by low resolutionsurveillance cameras or extremely wide aspect ratios where the patientarea is relatively small.

An alternative to facial feature recognition for assessing patientpositions is by using geometric element comparisons to the boundary ofthe head defined in step 1108. FIGS. 13A-13H, 15A and 150 are diagramsthat graphically illustrate the comparison process. Initially, it iswell known that in most cultures the relative dimensions of frontal andside views of a human face are relational. For example, a human head isapproximately two thirds as wide as it is high. Therefore, once aboundary for the patient has been defined, the width (w) of the facialarea is measured (shown as line 1304 in FIGS. 13A and 13E). From thatwidth w, upper facial box 1310 having dimensions w×w can be constructedfor constraining an ellipse 1320 to be fitted to the upper boundary ofthe patient's head. A lower facial box 1312 having dimensions w×w isaligned directly below box 1310. In practice, the dimensions of facialboxes 1310 and 1312 are actually w×w−sin θ, where θ is the angle betweenthe view axis and the facial plane (see FIGS. 138 and 13F). In casessuch as HCFs where the surveillance camera is not perpendicular to thefacial plane, the vertical height of the patient in the frame issomewhat compressed, (where θ≈30°, the patient's height halved). Withellipse 1320 aligned with the upper boundary of the head 1302, a secondellipse is aligned with the upper portion in lower facial box 1312,fitted totally within the facial boundary, in one of three candidatepositions: centered 1324, aligned right 1326 and aligned left 1322 (seeFIGS. 13C, 13G, 15A and 150). The lower ellipse has dimensionsw/2×w/2−sin θ, half the size of the upper ellipse 1320. The orientationof the patient's head and face is based on which of the three ellipsesfit within the facial boundary, if only centered ellipse 1324 fitswithin the facial boundary, then the face is oriented forward and thepatient is positioned on her back; if right ellipse 1326 fits within thefacial boundary, then the face is oriented toward the patient's left andthe patient is positioned on her left side; if left ellipse 1322 fitswithin the facial boundary, then the face is oriented toward thepatient's right and the patient is positioned on her right side (seeFIGS. 13C, 13G and 150). Finally, the proper fitting of the upper andlower ellipses also delineated approximate horizontal positions offacial features that is useful in optimizing scanning step 1116. Forinstance, brow line 1332 vertically bisects upper ellipse 1320 and isthe approximate location of the eye brows; eye line 1334 bisects thelower half of located upper ellipse 1320 and is the approximate lowerextent of the eyes; nose line 1335 is located at the intersection of theupper and lower ellipses and is the approximate lower extent of thenose; mouth line 1336 bisects the lower ellipses and is the approximatelocation of the mouth; and location of the eyes chin line 1338 islocated at the lower extent of the lower ellipses is the approximatelower extent of the chin. Finally, the location of the connection of theneck and shoulders can be approximated along lines 1340 between theintersection between the center line and brow line 1332, and eitherlower corner of lower facial box 1312. The width of the shoulders isbased on the orientation of the patient and width w; on the right orleft sides, the outer extent of the shoulders is approximately 1.5 w indiameter (see box 1342 in FIG. 13H); and on the back of the outer extentof the shoulders is approximately 2.0 w in diameter (see box 1342 inFIG. 130).

While facial feature recognition is a powerful tool in assessing theposition of a patient, the analysis technique has some drawbacks in anHCF setting, that is often the resolution of surveillance camera 216 isnot sufficient to distinguish contrast changes representing eyes 1204Land 1204R or ears 1206L and 1206R, or other distinguishing facialfeatures such as the patient's nose or mouth. One method of determiningwhich position the patient is in is to find the position of eachshoulder relative to the patient's neckline. A conventional video cameradoes not have stereoscopic vision, so it does not have the informationavailable to give it the same level of depth perception that human eyeshave. However, since the human body is symmetrical, an algorithm is usedto determine which shoulder is closer to the camera. Initially, camera216 is lined up with bed 321, so that the two sides of the bed begin andend at approximately the same vertical position in the camera's field ofview, then the image can be used to deduce the patient's position bylooking at the shoulders. The analysis technique for determiningpatient's position from the patient shoulders is described directlybelow.

FIG. 14 is flowchart depicting a process for assessing a patient'sposition by detecting the positions and orientation of the patient'sshoulders in a video frame in accordance with another exemplaryembodiment of the present invention. Essentially the process steps areidentical to those discussed in FIG. 11A, however, identifying thelocation of the patient's shoulders in step 1110 may also includetechniques for recognizing a pattern imprinted on the patient's hospitalgown on the shoulders. The pattern, as further depicted in FIGS.15A-15F, might be any clearly recognizable geometric shape, such ascircles 1512L and 1512R. Optimally, a mechanism should exist fordistinguishing the pattern imprinted on the left shoulder from thepattern imprinted on the right shoulder, for instance placing a distinctpattern on each shoulder. With respect to circles 1512L and 1512R, thetwo might have distinct coloration or color patterns, or alternatively,for example, the left shoulder may have a circle and the right atriangle, or parallel lines oriented vertically on one shoulder andhorizontally on the other, etc. In any case, the process of FIG. 11Acontinues by observing the vertical positions of the left and rightshoulder patterns (in FIG. 15A-15F, 1512L and 1512R), the verticalposition and orientation of shoulder line 1510 (step 1412). Next, thepatient's position can be assessed by one or several of the followingconditions being met (step 1414): patient is on the back if shoulderline 1510 is located well below the patient's face and is approximatelyvertical and/or left shoulder pattern 1512L is clearly distinguishableon the patient's left side and right shoulder pattern 1512R is clearlydistinguishable on the patient's right side (see FIGS. 15A and 150); orthe patient is oriented quarter turned to right if shoulder line 1510 ispositioned below face and lower on patient's right (approximately 45°)(see FIG. 15E); or the patient is oriented quarter turned to left ifshoulder line 1510 is positioned below face and lower on patient's left(approximately −45°) (see FIG. 158); or the patient is on the right sideif shoulder line 1510 is positioned above or near face and lower onpatient's right or if only left shoulder pattern 1512L is visible (seeFIG. 15F); or patient on left if shoulder line 1510 is positioned aboveor near face and lower on patient's right or if only right shoulderpattern 1512R is visible (see FIG. 15C); and finally, the patient isoriented to either the left or right side but which side is notdeterminable if shoulder line 1510 is undeterminable because only one ofshoulder patterns (1512L and 1512R) are visible, but not recognizablefrom the other of shoulder patterns (1512L and 1512R). Next, changes inthe patient's position can be assessed by comparing the patient'sposition in current NTM frame N with the previously saved NTM frame(step 1416). The process then reverts to step 1104 for a new frame N.

The principles discussed above are relevant even if the patient is lyingat a position that is skewed from the center of bed 321. This couldreverse the position of the shoulders in the camera's field of view.This is acceptable, however, because the aim of this analysis techniqueis to only detect changes in the patient's position.

Assessing the patient's position from the position and orientation ofthe patient's shoulders is highly advantageous because the amount offrame analysis can be greatly reduced over facial feature recognitionand because this analysis method lends itself to lower resolutioncameras. However, the presently described process may be even furtheroptimized by reducing the area to be analyzed within area of interest840 to only the respective previous positions of left shoulder pattern1512L and right shoulder pattern 1512R. This is possible by designatingleft shoulder area 1514L and right shoulder area 1514R for analysis inthe next NTM frame identified. If, in the next NTM frame the positionsof the patient's shoulders are found within left shoulder area 1514L andright shoulder area 1514R as in the previous NTM frame, then theanalysis can cease as the patient's position has not changed. Thisprocessing step (or perhaps pre-processing step) is graphicallyillustrated between FIGS. 15A and 158, showing right shoulder pattern1512R having moved from area 1514R to 1516R and similarly, left shoulderpattern 1512L having moved from area 1514L to 1516L; and again betweenFIGS. 158 and 15C, showing right shoulder pattern 1512R having movedfrom area 1516R to 1518R and similarly, left shoulder pattern 1512Lhaving moved from area 1516L to 1518L (assuming the left shoulderpattern is visible). The process is depicted for the right side betweenFIGS. 150 and 15E and again between FIGS. 15E and 15F.

With regard to recognizing patterns that correspond to patientpositions, one of the difficulties in tracking the patient's position onthe hospital bed using a video monitoring device is that often the lowerportion of the patient's body is obscured by a solid-colored sheet orblanket. This inhibits patient surveillance sub-system 220 fromdetecting contrast between the patient and bed 321, thereby preventingthe system from observing the position of the patient's lower body. Tosolve this problem, the presently described system and method for usinga video monitoring system to prevent and manage decubitus ulcers inpatients utilizes custom-printed sheets which contain a regularlyrepeating pattern (such as a grid, circles or other repeating geometricpattern that can be recognized by the system). This orientation of thispattern in the image frame will allow the camera to detect contours atdifferent positions in the bed sheet, and will allow the system to inferthe patient's position from those contours.

With regard to yet another analysis technique for assessing thepatient's position and/or assessing changes in the patient's position,it is possible to identify the patient's position from representativehorizontal patient positions taken at a discrete number of horizontalstripes within area of interest 840 or, alternatively, from theorientations between the respective representative horizontal patientpositions taken in the horizontal stripes. With regard to eithertechnique, the vertical positions of the respective horizontal stripeswithin area of interest 840 are predetermined to coincide with thevertical position of parts of the patient's body that are highlyindicative of the patient's position, for instance shoulders, head/neck,hips, knees, torso and feet. The vertical positions of these horizontalstripes in a generic area of interest 840 may be designated inpercentage increments from the top of area of interest 840; the topbeing 00%, to the bottom, being 100%, with the vertical middle of areaof interest 840 being the 50% horizontal stripe (see the examples inFIGS. 18A-18C diagrammatically illustrating area of interest 840superimposed on patient bed 321 with five horizontal stripes, where 17%stripe 01 corresponds to the patient's head, 29% stripe 03 correspondsto the patient's shoulders, 56% stripe 05 corresponds to the patient'ships, 72% stripe 07 corresponds to the patient's knees and 86% stripe 09corresponds to the patient's feet). Alternatively, the verticalpositions of the horizontal stripes may be designated as percentageincrements of the patient, from the top of the patient's head to thebottom of the patient's foot. In so doing, the vertical positionpercentages are relational to the size of the patient and not the sizeof the area of interest. In any case, a single horizontal patientposition along any discrete horizontal stripe may be selected within aline interval defined by using any of the analysis techniques describedabove, for instance by grouping color temperatures along the stripe, byfinding the left and right extents of the patient interval along thestripe using contrast or alternatively, by using a representativehorizontal position such as the median of an interval that deviates froman average luminance value by a predetermined amount for each horizontalstripe. The following algorithms describe methods for using a curveplotting horizontal position against pixel luminance to describe theposition of the patient. Optimally, each uses a plotted curve showingthe luminance of pixels as they range horizontally from one side of thearea of interest 840 containing patient's bed 321, to the other side ofthe area of interest 840. Since the distance of the object from thelight source (room light during the day, and infrared LEDs from thecamera at night) affects the amount of light which is reflected to thecamera, the luminance measurements can help describe the height of thevarious portions of the patient on the bed. Only luminance techniquewill be described immediately below, however, acceptable results may beobtained using other techniques disclosed here within without departingfrom the scope of the invention.

FIGS. 16A and 16B are flowcharts depicting a process for assessingchanges in a patient's position by detecting parts of a patient's bodyrepresented in a video frame that lie along one of a plurality ofdiscrete horizontal lines in accordance with an exemplary embodiment ofthe present invention. The present example depicts the patient in arecumbent position on patient bed 321 for simplicity and is not intendedto limit the scope of the present invention. The patient may be, inaddition to recumbent in a bed, seated in a chair or wheelchair or insome other position at, for example, a medical diagnostic or therapyapparatus. In accordance with this analysis process, the position of apatient's body can be described as a plurality of representativehorizontal positions, one for each of the discrete horizontal lines. Theprocess can be optimized by limiting the portion of the image frame toan area of interest that includes the patient. With regard to a specificexample, assuming again the area of interest is 64×84 pixels of the viewframe of camera 216. then, each of the five exemplarily horizontalpositions would occur between pixel 0 and pixel 63 pixel of thehorizontal line. See, for example, positions 1801, 1803, 1805, 1807 and1809 for position 1800 depicted in FIG. 18A, positions 1811, 1813, 1815,1817 and 1819 for position 1810 depicted in FIG. 18B and positions 1821,1823, 1825, 1827 and 1829 for position 1820 depicted in FIG. 18C. Thesepositions approximate the patient's position along the horizontal lines(for instance the 17% horizontal stripe 01, 29% horizontal stripe 03,56% horizontal stripe 05, 72% horizontal stripe 07 and 86% horizontalstripe 09) (in the figures, stripes are labeled by odd element numbersto correspond with position element numbers). Typically, area ofinterest 840 will be displayed on video monitor/touchpad 267 (forinstance as Virtual BedRails) however the horizontal stripes arerepresented in the figures for describing the present invention and aretypically not displayed after exiting the setup phase). In any case, theanalysis process determines the placement of the position along eachhorizontal line.

In accordance with one exemplary embodiment of the present invention therepresentative horizontal position of a particular horizontal line maybe designated as the median of a pixel interval having a colortemperature (optimally luminance) above the average color temperature ofthe particular horizontal line by some predetermined amount, percentageor ratio. The representative position may also be designated as someother predetermined point along the pixel interval. Hence, the patient'sposition in each image frame can be described by a plurality ofpositions along the horizontal lines at the respective verticalpositions. It is expected that at least three position points arenecessary, although in certain situations fewer positions may suffice todescribe the patient's position. In the present example the patient'sposition along five horizontal lines are interrogated. The descriptionof these patient positions can be compared between image frames todetermine if the patient's position has changed (see the table in FIG.19 that depicts the horizontal positions at the five discrete horizontallines 17% stripe 01, 29% stripe 03, 56% stripe 05, 72% stripe 07 and 86%stripe 09, for patient positions 1800, 1810 and 1820).

Turning to the flowchart, the process begins by camera control device210 receiving a description of the patient area (area of interest 840)which is manually input by the HCF staff, typically on videomonitor/touchpad 267 (step 1602). Internally, camera control device 210subdivides area of interest 840 into percentages, which also uses areaof interest 840 as the horizontal extent of the interrogation intervalusing this method. The plurality of vertical positions in the area ofinterest are received (step 1604). Each of these vertical positionsgenerally corresponds to a strategically positioned horizontal stripethat corresponds to a part of the patient's body that is particularlyindicative of the patient's position, e.g. head/neck, shoulders, torso,hips, legs, feet, etc. (step 1604). Only these vertical positions willbe interrogated for content within area of interest 840 of the imageframe. Therefore, their selections within the image frame shouldaccurately define the vertical position of the corresponding body partwithin the area of interest. The vertical positions within area ofinterest 840 are determined in advance and may be based on a generic setof vertical positions that generally correspond to every patient, ormore preferably might instead be based on factors common to only thepatient being monitored. These common factors include, for instance, theheight of the patient, her current position relative to the top (orbottom) of area of interest 840 and/or patient bed 321, etc. Optimally,the vertical positions may be selected by the HCF staff at patientmonitoring sub-system 270 that is unique for each patient beingmonitored for changes in position.

With the setup phase completed, the process receives frame N (step 1606)and scans the horizontal pixel rows at each of the vertical positions inframe N for predetermined pixel characteristics (step 1608). Asdiscussed previously, the pixel characteristics selected for use may beany optical characteristic capable of distinguishing the patient,however, optimally, luminance values are scanned for the reasonsdiscussed above and below. An average luminance value is determined foreach of the sets of luminance values along the horizontal pixel rows andposition points along each row that deviates from the average luminanceof that row by a predetermined percentage are identified (step 1610). Asdiscussed previously, it is expected that the luminance value willincrease with the elevations on the patient's bed, therefore, areaswithin the image frame with the patient, with correspondingly higherelevations, will tend to exhibit higher luminance values in the imageframe. Discriminating these values is optimized by analyzing pixelsacross horizontal rows because the light reflected off the patient areais relatively constant for like elevations along horizontal lines. Thisis due to the orientation of the patient's bed to the room light sourceand their light source of the video camera (see the discussion betweenFIGS. 5 and 6). Hence, the patient, if present, will appear along anyhorizontal pixel row in the video frame as an interval of pixel pointshaving luminance values above the average luminance for that horizontalpixel row, usually by some predetermined amount, for example by apredetermined ratio or percentage (this is represented in FIGS. 18A-18Cas an interval of shading along the five discrete horizontal stripes). Amedian horizontal position for the points having the above averageluminance for the horizontal pixel row is determined for each of thevertical positions (see positions 1801-1809 for patient position 1800 inFIG. 18A, positions 1811-1819 for patient position 1810 in FIG. 188 andpositions 1821-1829 for patient position 1820 in FIG. 18C) (step 1612).Although the concept is of little significance for this method, itshould be appreciated that each median position corresponds to a vertexof a patient position representation curve that describes the patient'sposition (depicted in the figures as a dashed line connected to thepatient position points). However, with regard to the presentlydescribed embodiment, only the absolute horizontal positions of themedians is used to determine patient movement.

In any case, frame N is tested for NTM (step 1614), for instance bycomparing the absolute positions of the medians at the verticalpositions determined for frame N with the median positions forcorresponding vertical positions determined for frame (N−1). If the twocorresponding sets of horizontal positions differ, frame N containstemporal motion, is flagged (step 1616) and not used for determiningchanges in the patient's position. Frame N is flagged for deletion atcleanup (step 1618) and the process then returns to step 1606 for thenext frame N.

If, however, at step 1614, no temporal motion is detected, frame N isflagged as an NTM frame (step 1620) and the median positions at thevertical positions of frame N are compared with median positions for thecorresponding vertical positions from the last saved NTM frame (N−1)(step 1622). The comparison between the median positions of the twoimage frames can be based on a predetermined difference betweencumulative medial position values (for example a cumulative change inthe median positions of fifteen pixel positions) or a predetermineddifferent value for each vertical position (for example a medianposition of five pixel positions for the 17% horizontal stripe 01 and/orsix pixel positions for the 29% horizontal stripe 03, and/or four pixelpositions for the 56% horizontal stripe 05 and so on). It is expectedthat whether the patient is recumbent or seated will bear on thesensitivity between images necessary to demonstrate a change in thepatient's position, e.g., greater differences between the medialpositions are needed to demonstrate a change in recumbent patient'sposition in comparison to a seated patient.

If the two corresponding sets of corresponding median positions areidentical (or within a predetermined amount), the patient's position isunchanged and frame N contains no useful decubitus ulcer procedure eventinformation. Frame N is then flagged for deletion (step 1618). Theprocess then returns to step 1606 for the next frame N.

If, at step 1622, the median positions from frame N have changed fromthe last NTM frame by the predetermined amount, patient motion isinferred in the current frame N and that frame is annotated as havingpatient movement and time stamped (step 1624). Frame N is also flaggedfor saving (step 1626) and the process then returns to step 1606 for thenext frame N. As discussed elsewhere above, the magnitude of thepatient's movement may also infer a change in the position of thepatient, in addition to mere patient movement. Clearly, the presentprocess is intended to automatically alert the HCF staff of patientmovement for attention/intervention by the HCF staff. However, if theamount of patient movement detected exceeds a second, higher thresholdlevel, it might be inferred that the patient has changed positions (seefor instance step 622 in FIG. 6A). In that case, nurse monitor device260 might instead issue a higher level alert, such as an alarm, thatsignifies more immediate attention may be required by the HCF staff.

While the process described immediately above is a particularly elegantanalysis technique, requiring extremely low system resources for rapidlyevaluating as few as three pixel rows, the process generally cannotidentify a patient's position. One shortcoming associated with thepresently described analysis technique is that it utilizes absolutehorizontal positions of the discrete vertical positions that do notdefine or correspond to a particular patient position. However, thecharacter of the patient position representation curve, comprised of theseparate horizontal positions between the discrete vertical positions,may itself be indicative of patient positions. Hence, it is advantageousto describe the patient position representation curve with relativeattributes rather than absolute attributes such as positionalinformation. In so doing, a library of patient position representationcurves can be determined in advance, each position representation curvecorresponding to a specific patient position, that can be used forcomparisons with a patient position curve constructed from a particularimage frame.

One relative attribute used to describe is the orientation betweenposition point vertices, or the line segment between vertices of thepatient position representation curve. For the purposes herein, theorientations will be described as the angular orientation ϕv of any linesegment V between a vertex and an adjacent, lower vertex (i.e.,270°>ϕv<ϕ_(v)>90°, although the use of other notations are possible. Forinstance, see the library of patient positions illustrated in FIG. 20depicting a plurality of patient position representation curvesrepresented by line segments at angular orientations ϕv between the fiveexemplary vertical positions. For example, patient positionrepresentation curve 2012 describes the patient on her back, patientposition representation curve 2014 describes the patient on her leftquarter, patient position representation curve 2016 describes thepatient on her left side, patient position representation curve 2018describes the patient on her right quarter and patient positionrepresentation curve 2020 describes the patient on her right side. Theangular orientations ϕv of these line segments can be tabulated (such asdepicted in FIG. 21) for each patient position representation curve ofthe library. For instance, row 2102 contains the angular orientations ϕof line segment V₁₋₂ between the highest and second highest vertices forpatient positions 2012, 2014, 2016, 2018 and 2020, row 2104 contains theangular orientations ϕ of line segment V₂₋₃ between the second and thirdhighest vertices, row 2106 between the third and fourth highest verticesand row 2108 between the fourth and fifth highest vertices for patientpositions 2012, 2014, 2016, 2018 and 2020. With the library of patientpositions represented by angular orientations between verticalpositions, the process can easily recognize a patient's position bycomparing the angular orientations between vertical positions from animage frame with those in the library.

FIGS. 17A and 17B are flowcharts depicting a process for assessing apatient's position by representing the patient's body, from arepresentation in a video frame, as a patient position curve comprisingline segments oriented from each other at particular orientations thatmay be referenced to a library of patient position angular orientationsin accordance with an exemplary embodiment of the present invention.This technique is also described with respect to a patient in arecumbent position on patient bed 321. The use of a recumbent patient ismerely to simplify the discussion and is not intended to limit the scopeof the present invention. The patient may be, in addition to recumbentin a bed, seated in a chair or wheelchair or in some other position at,for example, a medical diagnostic or therapy apparatus. Beforeproceeding it should be mentioned that even though the present processanalyzes patient positional information in a video frame using onlyrelative attributes, it is possible that the description of thepositional curve is undefined in the library. In that case, the processmerely reverts to a description of the positional curve using absolutemedian position information for assessing changes in the patient'sposition depicted in FIG. 16B. The distinctions will become apparentwith the description. The process begins in setup phase by cameracontrol device 210 receiving a description of area of interest 840 (step1702) and plurality of vertical positions in the area of interest arereceived (step 1704), both discussed with specificity at steps 1602 and1604 above. However, here it should be emphasized that the selection ofthe vertical positions should correlate to the vertical positions usedfor compiling the library of patient positions discussed below. What isimportant is the part of the patient body being monitored by thehorizontal stripe and not the absolute vertical position of the stripewithin the area of interest. The limitations of this selection will bediscussed further below.

Next, the process receives frame N (step 1706) and scans the horizontalpixel rows at each of the vertical positions in frame N for luminancevalues for the pixels in the horizontal row (step 1708), as alsodiscussed immediately above. As before, an average luminance value isdetermined for each horizontal pixel row and position pixel points alongeach row that deviates from the average luminance of that row by apredetermined percentage are identified (step 1710). Finally, a medianhorizontal position for the points having the above average luminancefor the horizontal pixel row is determined for each of the verticalpositions (step 1712). Although the presently described process does notutilize the median horizontal position information, it will be retainedif it is later determined that frame N contains decubitus ulcerprocedure event information in case the patient's position cannot beidentified (see below step 1726).

From here, the process converts the absolute position points for each ofthe vertical positions to the angular orientation between positionpoints (or of the line segment between the points) (step 1714). Theprocess measures the angular relationships, or angular orientation ϕvbetween position points in adjacent vertical positions, or the linesegment V extending between the adjacent position points (see again thelibrary of patient positions illustrated in FIG. 20 which are tabulatedas depicted in FIG. 21). The aim here is to describe a patient'sposition as a sequence of angular relationships between parts of thebody that accurately describe the patient's position. With regard to theexample, angular orientation V₁₋₂ is measured between the first andsecond vertical positions, head/neck (horizontal stripe 01@17° and theshoulder (horizontal strip 03@29°, another angular orientation V₂₋₃ ismeasured between the second and third vertical positions, the shoulders(horizontal strip 03@29° and hips (horizontal strip 05@56° and so on.

Next, frame N is tested for NTM by comparing the angular relationshipsbetween the vertical positions for frame N with the angularrelationships between the corresponding vertical positions for frame(N−1) (step 1716). If the corresponding angular orientations havechanged between frames, frame N is flagged as having temporal motion(step 1718) and flagged for deletion (step 1720). The process thereverts to step 1706 for a new frame N.

If, at step 1716 no motion is detected, frame N is flagged for NTM (step1722). Next, the patient's current position is identified by comparingthe angular relationships for frame N with a library of angularrelationship/body position data (step 1724) (see table of angularrelationship/body position data depicted in FIG. 21). Because thepresent process describes body position in relative attributes, theattributes may be interpreted from a comparison to a library of similarattributes. The library of angular relationship/body position datadescribes body positions by angular relationships between parts of thebody, for example between the head/neck and shoulders, or between theknees and feet, etc. Comparisons to these angular relationships are notconstrained by the absolute positions of the position points from whichthe relations are gathered, or even the absolute dimension of the areaof interest or the height of the patients being monitored. The librarycan be assembled from archived surveillance video by simply identifyingpatient positions in the video and interrogating the frame for theangular relationships. The data may be compiled, for example, byaveraging the corresponding angular relationship data from identicalpatient positions over a time period, between different archival videos,or both. It is possible that for each angular relationship, acorresponding angular tolerance exists. In that case, if angularrelationships from a particular video frame matches the correspondingangular relationships of a library position, within the tolerances, theposition is matched. Using whatever tolerances, if a match is identifiedfrom the library, the patient's position if frame N is identified (step1728) and can be compared with the patient's position last saved NTMframe (step 1730). If the patient positions in the two frames match,then the patient may have moved, but did not change position. In thatcase, frame N contains no decubitus ulcer procedure event information inaddition to the previously saved NTM frame, and therefore frame N can beflagged for deletion (step 1720) and the process reverts to step 1706for a new frame N. If, at step 1730 the patient's position from the twoframes do not match, then the patient has changed positions. Frame N isthen annotated as having a decubitus ulcer procedure event, i.e., achange in the patient's position, along with the patient's new positionand being timestamped (step 1732) and is flagged for saving (step 1734).The process then reverts to step 1706 for a new frame N.

Returning to step 1726, there may be cases where the patient's positionin frame N cannot be identified in the library of body positions, thatis, the angular relationships derived in frame N do not match any of theknown corresponding angular relationships for a body position. In thosecases, the process cannot continue on the basis of comparing thepatient's position between frames for changes and must revert to usingthe medial position to determine patient movement, hence the processcontinues at step 1622 of the process depicted in the flowchart of FIG.16B. Once frame N has been interrogated, the process reverts to step1706 as before.

While it is understood from the descriptions of the generic processes ofthe flowcharts depicted in FIGS. 5, 6A and 6B, the process performsadditional functions that were omitted from the flowcharts depicted inFIGS. 9A, 9B, 11A, 118, 14, 16A, 6B, 17A and 178 for clarity, such asmonitoring the time with regard to the turn schedule, alerting the HCFstaff to decubitus ulcer procedure events, requesting verifications,etc. Furthermore, the process continually identifies video frames withuseful decubitus ulcer procedure event information that should be savedas part of the patient position history. In addition to the decubitusulcer procedure events discussed, the present system and method forusing a video monitoring system to prevent and manage decubitus ulcersin patients may be configured to memorialize video frames of otherinformation and events, such as pre-event frames, post-event frames, HCFstaff intervention events, patient repositioning alert events and timedsample events.

The presently described system and method for using a video monitoringsystem to prevent and manage decubitus ulcers in patients greatlyreduces the amount of manual intervention required from an attending HCFstaff member by automatically detecting decubitus ulcer procedureevents, identifying the events as being one or all of patient movement,a change in the patient's position or a patient turn event. The systemindependently documents patient movement, positions, changes in positionand turning as a succinct video log containing only relevant video,without manual intervention from the HCF staff. Additionally, the videorecord is automatically annotated with relevant decubitus ulcerprocedure event information, timestamps, HCF staff comments andverifications and archived to locations secure from unauthorizedalterations. These processes run largely in the background until andunless the system determines that intervention by the HCF staff iswarranted.

Simultaneously, the presently described system and method for using avideo monitoring system to prevent and manage decubitus ulcers inpatients provides the HCF staff with an easily understood visualinterface at patient monitoring system 270 for interpreting decubitusulcer procedure event information receiving alerts, alarms and warnings,as well as inputting data and responding to verification requests andother system queries. FIGS. 22A, 22B and 22C are three screenshots ofthe video monitoring system in response to different decubitus ulcerprocedure events. These screens are merely exemplary for understandingthe interface. In normal operating mode, video monitor/touchpad 267,connected to Nurse monitor device 260 displays surveillance videos froma plurality of patient rooms in miniature windows in a storyboardpresentation (not shown) with one large window showing an enlarged viewof a selected room (see FIGS. 22A, 22B and 22C). Additionally, theresolution of a selected patient room can be greatly enhanced,temporarily, by expanding the window over video monitor/touchpad 267.FIG. 22A depicts a typical surveillance window 2250 in which patientroom 2252 is presented. Within room 2252 is patient bed 321 and alignedon the bed, area of interest 840 is delineated with a colored boundary.In run mode, without an active decubitus ulcer procedure event, certaindecubitus ulcer procedure event information is displayed for the HCFstaff in an abbreviated format. For instance, icons 2204 show theprocesses that are currently running, such as event alert/alarmprocesses, event warning processes, turn schedule monitoring process andnavigation icons for reducing/expanding surveillance window 2250. Inaddition, within surveillance window 2250, substantive informationrelated to the state of the turn schedule is displayed in time-to-turnfield 2202, for example a dropdown box. As will be discussed below,surveillance window 2250 provides the means to access additionaldecubitus ulcer procedure event information as well as editing orverifying the information. Note also that the present inventionrecognizes that in certain situations the video data outside area ofinterest 840 will be analyzed for motion, such as for detecting thepresence of the HCF staff in patient room 2252. Portions of patient room2252 that tend to give erroneous or conflicting analysis results can beselectively omitted from processing by bounding the area to be omitted,such as area 2206 containing balloons that constantly move with the aircurrents within patient room 2252.

Because the attending HCF staff is charged with caring for severalpatients in different rooms, the system actively alerts the HCF staff toan event, visually and audibly, identifies the patient room by, forexample, highlighting border 2208 of surveillance window 2250 andanimating alarm icon 2214 to draw the attention of the HCF staff to thatwindow and that particular patient room (see FIG. 22B). Additionally,decubitus ulcer procedure event information 2210 is prominentlydisplayed within surveillance window 2250, for instance the HCF staffmanual intervention for repositioning the patient in room 2252. At theconclusion of the event, the system recognizes that the patient has beenrepositioned consistent with the turn schedule and resets the timer inaccordance with the new position in the turns schedule (see FIG. 22C).Additionally, the HCF staff can access addition information from, forexample, field 2202, depicted opened as dropdown box 2220 in FIG. 22C.There, several useful data items are displayed, including time-to-turnfield 2202, next patient position field 2224, current patient positionfield 2226 with patient position selection menu 2228 showing thesequence of patient positions, chronologically arranged, from thepatient order. Also provided is button means 2230 for the HCF staff toverify the current event state of the process.

Finally, the presently described system and method for using a videomonitoring system to prevent and manage decubitus ulcers in patientsprovides the HCF staff with a heretofore unknown mechanism of rapidlyassessing the state of any patient and reviewing temporal events inchronological order merely by actuating patient position history reportbutton 2212.

In addition to the heavy load a nursing staff faces, the nursing staffchanges shifts twice per day, resulting in a new caregiver for eachpatient. The outgoing staff must exchange all relevant information withthe incoming staff so that the care schedule for the patients is notinterrupted. This is particularly important with prevention of decubitusulcers. If the outgoing staff forgets to let the incoming staff knowthat the patient is due to be moved within the next 15 minutes, it'spossible that the patient will go 60-90 minutes without attention fromthe HCF staff. Since the presently described system and method for usinga video monitoring system to prevent and manage decubitus ulcers inpatients creates and saves a video history of the patients in at leastLocal surveillance sub-system 220, the system can be leveraged to offerassistance to the incoming HCF staff in the form of an instant visualrecord of the previous few hours. Rather than rely on a verbal recordfrom the outgoing staff, the incoming staff can click position historyreport button 2212 on the user interface and/or window 2250, and thesystem will respond with screenshots of the patient, chronologicallyordered from a rolling time interval determined by the HCF staff (seethe rolling time interval selection 709 for a historical position reportin FIG. 7). These images include at least frames of the patient beforeand after a repositioning event. The images may also contain annotationsof decubitus ulcer procedure event information. The video history mayalso include images of all decubitus ulcer procedure events thatoccurred within the rolling time interval, alert events, sample framestaken at predetermined time intervals and the like.

With further regard to creating, compiling and presenting a videohistory of patient decubitus ulcer procedure events, the flowchartpresented in FIG. 23 depicts one exemplary method creating video historyof patient decubitus ulcer procedure events, while the flowchartpresented in FIG. 24 depicts an exemplary method compiling the videohistory of patient decubitus ulcer procedure events and presenting thevideo history to the HCF staff as a chronologically ordered set ofscreenshots of the patient in accordance with the present invention.Initially, the HCF staff identifies which events should be included inthe video record of patient decubitus ulcer procedure events (step2302). Typically, the system will select certain decubitus ulcerprocedure events for the record without intervention from the HCF staff.These may include turn schedule events, patient repositioning events,HCF staff manual interventions, etc., but may also include pre-event andpost-event frames and the selection frequency for timed sample events.The HCF staff may also configure the rolling patient position historytime for compiling the patient record. The rolling time selection shouldbe long enough for the HCF staff unfamiliar with the patient toaccurately assess the state of the patient decubitus ulcer procedure andidentify any immediate HCF staff interventions that should be performed.Next, the HCF staff configures the system with the turn schedule (step2304). The turn schedule data provides the system with the turn intervaltimes and corresponding positions necessary for making complex eventdecision. The system then monitors the surveillance area including thepatient (step 2306) and automatically detects events by (step 2308), forexample, analyzing the video frames and monitoring intervention timeintervals with regard to the turn schedule. Video frames that representpatient decubitus ulcer procedure events are annotated with eventinformation and timestamped (step 2310). All video frames that have beenannotated with event information and/or timestamped are retained in atleast a local memory of camera control device 210 (step 2312). Thoseframes constitute the video record of the patient position history, butmay be retained in an ad hoc sequence of video frames representative ofmany patient, medical and security procedures. It is expected that anypatient position history will contain at least a chronological sequenceof image frame depicting the patient's current position at eachscheduled turn event (pre-event frame). Alternatively, a more completepatient position history may also contain additional image frames aftereach HCF staff manual intervention depicting the patient's new positionafter a turn. Still further, the patient position history may alsocontain timed sample frames of the patient's position at predeterminedtime intervals and image frames showing the patient's position at anydecubitus ulcer procedure event that is automatically detected bypatient movement monitoring system 200. Finally, the patient positionhistory may also contain images frames of patient positions that havebeen manually selected by the HCF staff at nurse monitor device 260 for,example, depicting some decubitus ulcer procedure event informationconcerning the patient's position.

In response to a command from the HCF staff, such as actuating patientposition history report button 2212, the system compiles the patientposition record from the various video frames retained in memory andinvokes a presentation process. Essentially, the system, typicallypatient monitoring system 270, requests only the images framespertaining to a particular procedure or event from local surveillancesub-system 220, such as patient decubitus ulcer procedure events. Localsurveillance sub-system 220 usually has possession of the most recentfew hours of video data and searches its memory for video framesannotated as being representative of the events over the pre-selectedrolling time period (step 2402). The individual image frames arecompiled into a chromatically ordered history of patient positionscreenshots showing all relevant events that occurred during the timeperiod (step 2404). Finally, the presentation process displays thechromatically ordered history in a manner most conducive to the HCFstaff understanding the current state of the patient decubitus ulcerprocedure (sep 2406). One presentation is for the HCF staff to merelyscan through the screenshot individually, one at a time. Alternatively,the system may present multiple screenshots, in chronological order, asthumbnail images or in miniature display windows. Typically, anydecubitus ulcer procedure event information annotated with an imageframe will be displayed to the HCF staff with the image frame. The HCFstaff can then visually scan the screenshots and expand any that needclarification. Once one set of screenshots have been reviewed, the HCFstaff calls for the next set until all images in the rolling time periodhave been reviewed.

As will be appreciated by one of skill in the art, the present inventionmay be embodied as a method, system, or computer program product.Accordingly, the present invention may take the form of an entirelyhardware embodiment, an entirely software embodiment (includingfirmware, resident software, micro-code, etc.) or an embodimentcombining software and hardware aspects all generally referred to hereinas a “circuit” or “module.” Furthermore, the present invention may takethe form of a computer program product on a computer-usable storagemedium having computer-usable program code embodied in the medium.

Any suitable computer readable medium may be utilized. Thecomputer-usable or computer-readable medium may be, for example but notlimited to, an electronic, magnetic, optical, electromagnetic, infrared,or semiconductor system, apparatus, device, or propagation medium. Morespecific examples (a nonexhaustive list) of the computer-readable mediumwould include the following: an electrical connection having one or morewires, a portable computer diskette, a hard disk, a random access memory(RAM), a read-only memory (ROM), an erasable programmable read-onlymemory (EPROM or Flash memory), an optical fiber, a portable compactdisc read-only memory (CD-ROM), an optical storage device, atransmission media such as those supporting the Internet or an intranet,or a magnetic storage device. Note that the computer-usable orcomputer-readable medium could even be paper or another suitable mediumupon which the program is printed, as the program can be electronicallycaptured, via, for instance, optical scanning of the paper or othermedium, then compiled, interpreted, or otherwise processed in a suitablemanner, if necessary, and then stored in a computer memory. In thecontext of this document, a computer-usable or computer-readable mediummay be any medium that can contain, store, communicate, propagate, ortransport the program for use by or in connection with the instructionexecution system, apparatus, or device. The computer-usable medium mayinclude a propagated data signal with the computer-usable program codeembodied therewith, either in baseband or as part of a carrier wave. Thecomputer usable program code may be transmitted using any appropriatemedium, including but not limited to the Internet, wireline, opticalfiber cable, RF, etc.

Moreover, the computer readable medium may include a carrier wave or acarrier signal as may be transmitted by a computer server includinginternets, extranets, intranets, world wide web, ftp location or otherservice that may broadcast, unicast or otherwise communicate anembodiment of the present invention. The various embodiments of thepresent invention may be stored together or distributed, eitherspatially or temporally across one or more devices.

Computer program code for carrying out operations of the presentinvention may be written in an object oriented programming language suchas Java?, Smalltalk or C++. However, the computer program code forcarrying out operations of the present invention may also be written inconventional procedural programming languages, such as the “C”programming language. The program code may execute entirely on theuser's computer, partly on the user's computer, as a stand-alonesoftware package, partly on the user's computer and partly on a remotecomputer or entirely on the remote computer. In the latter scenario, theremote computer may be connected to the user's computer through a localarea network (LAN) or a wide area network (WAN), or the connection maybe made to an external computer (for example, through the Internet usingan Internet Service Provider).

A data processing system suitable for storing and/or executing programcode may include at least one processor coupled directly or indirectlyto memory elements through a system bus. The memory elements can includelocal memory employed during actual execution of the program code, bulkstorage, and cache memories which provide temporary storage of at leastsome program code in order to reduce the number of times code must beretrieved from bulk storage during execution.

Input/output or 1/0 devices (including but not limited to keyboards,displays, pointing devices, etc.) can be coupled to the system eitherdirectly or through intervening 1/0 controllers.

Network adapters may also be coupled to the system to enable the dataprocessing system to become coupled to other data processing systems orremote printers or storage devices through intervening private or publicnetworks. Modems, cable modem and Ethernet cards are just a few of thecurrently available types of network adapters.

The exemplary embodiments described below were selected and described inorder to best explain the principles of the invention and the practicalapplication, and to enable others of ordinary skill in the art tounderstand the invention for various embodiments with variousmodifications as are suited to the particular use contemplated. Theparticular embodiments described below are in no way intended to limitthe scope of the present invention as it may be practiced in a varietyof variations and environments without departing from the scope andintent of the invention. Thus, the present invention is not intended tobe limited to the embodiment shown, but is to be accorded the widestscope consistent with the principles and features described herein.

The flowchart and block diagrams in the Figures illustrate thearchitecture, functionality, and operation of possible implementationsof systems, methods and computer program products according to variousembodiments of the present invention. In this regard, each block in theflowchart or block diagrams may represent a module, segment, or portionof code, which comprises one or more executable instructions forimplementing the specified logical function(s). It should also be notedthat, in some alternative implementations, the functions noted in theblock may occur out of the order noted in the figures. For example, twoblocks shown in succession may, in fact, be executed substantiallyconcurrently, or the blocks may sometimes be executed in the reverseorder, depending upon the functionality involved. It will also be notedthat each block of the block diagrams and/or flowchart illustration, andcombinations of blocks in the block diagrams and/or flowchartillustration, can be implemented by special purpose hardware-basedsystems which perform the specified functions or acts, or combinationsof special purpose hardware and computer instructions.

The terminology used herein is for the purpose of describing particularembodiments only and is not intended to be limiting of the invention. Asused herein, the singular forms “a”, “an” and “the” are intended toinclude the plural forms as well, unless the context clearly indicatesotherwise. It will be further understood that the terms “comprises”and/or “comprising,” when used in this specification, specify thepresence of stated features, integers, steps, operations, elements,and/or components, but do not preclude the presence or addition of oneor more other features, integers, steps, operations, elements,components, and/or groups thereof.

What is claimed is:
 1. A method for using a video monitoring system toprevent and manage decubitus ulcers in patients, the method comprising:receiving a decubitus ulcer patient repositioning schedule at the videomonitoring system via a user interface including a plurality of patientposition selections and corresponding timer selections, the patientrepositioning schedule comprising a plurality of patient positions on asupport surface, the plurality of patient positions respectivelycorresponding to a plurality of different at-rest body orientations onthe support surface and a plurality of patient position time intervalsassociated with the plurality of different at-rest body orientations;receiving a plurality of image frames from a surveillance camera at thevideo monitoring system, each of the plurality of image framescontaining a patient on a support structure; overlaying one or morepixel lines on the plurality of image frames at a plurality of verticalpositions based on a position of the patient, the one or more pixellines described as pixels at given horizontal positions within theplurality of vertical positions of the plurality of image frames;scanning the one or more pixel lines at the plurality of verticalpositions for luminance values; determining median horizontal positionsof the one or more pixel lines based on pixels having above averageluminance within the luminance values; monitoring the position of thepatient on the support surface by determining which of the plurality ofdifferent at-rest body orientations the patient exhibits in theplurality of image frames by analyzing the plurality of image frameswith the video monitoring system based on the median horizontalpositions of the one or more pixel lines; determining the time in whichthe patient remains in a first patient position with the videomonitoring system based on the monitoring, the first patient positioncorresponding to one of the plurality of different at-rest bodyorientations; issuing a first alert within a surveillance windowinterface of the video monitoring system that indicates a correctpatient position and an indication that the patient is not in thecorrect patient position based on the patient remaining in the firstpatient position for a duration exceeding one of the plurality ofpatient position time intervals that is associated with the firstpatient position, the surveillance window interface including timeremaining in the correct patient position, a next patient positionaccording to the patient repositioning schedule, and a field to submit averification of a current position of the patient; detecting a patientrepositioning event with the video monitoring system based on themonitoring; determining whether the patient repositioning event is notin compliance with the patient repositioning schedule with the videomonitoring system based on whether the repositioning event is anunscheduled event and the position of the patient following the patientrepositioning event is in conflict with the patient repositioningschedule; and issuing a second alert within the surveillance windowinterface of the video monitoring system that indicates the correctpatient position and the indication that the patient is not in thecorrect patient position based on the patient repositioning event notbeing in compliance with the patient repositioning schedule.
 2. Themethod of claim 1, wherein detecting the patient repositioning eventcomprises detecting a transition from a second patient position of theplurality of patient positions to a third patient position of theplurality of patient positions.
 3. The method of claim 1, wherein thepatient repositioning schedule comprises an order for which the patientis to turn between the plurality of patient positions.
 4. The method ofclaim 3, wherein determining whether the patient repositioning event isnot in compliance with the patient repositioning schedule comprisesdetermining whether the patient has turned out of the order.
 5. Themethod of claim 1, wherein determining whether the patient repositioningevent is not in compliance with the patient repositioning schedulecomprises determining whether the patient has turned prematurely basedon one of the plurality of patient position time intervals.
 6. Themethod of claim 1, further comprising time stamping and storing at leastsome of the plurality of image frames in memory, wherein the patient isshown in each of the plurality of patient positions in the at least someof the plurality of image frames.
 7. The method of claim 1, whereinmonitoring the time in which the patient remains in the first patientposition and detecting the patient repositioning event are eachperformed by comparing at least one attribute of different image framesof the plurality of image frames between the different image frames. 8.The method of claim 1, wherein the plurality of patient position timeintervals are of different durations.
 9. The method of claim 1, whereineach of the plurality of patient positions on the support surface iseither of a recumbent position or a seated position.
 10. The method ofclaim 1, wherein the plurality of patient positions comprise a left sideposition, a right side position, and a back side position.
 11. Themethod of claim 1, wherein issuing one or both of the first alert andthe second alert comprises generating one or more of an audible alarmand a visual alarm.
 12. The method of claim 1, wherein the decubitusulcer patient repositioning schedule is one or more of entered, updated,or verified at the video monitoring system by a user.
 13. A system forvideo monitoring to prevent and manage decubitus ulcers in patients, thesystem comprising: a video camera configured to generate a plurality ofimage frames of a patient on a support structure; an interface includinga plurality of patient position selections and corresponding timerselections; and a data processing system having a processor connectedwith memory and configured to: receive a decubitus ulcer patientrepositioning schedule, the patient repositioning schedule comprising aplurality of patient positions on the support surface, the plurality ofpatient positions corresponding respectively to a plurality of differentat-rest body orientations on the support surface and respectively to aplurality of patient position time intervals associated with theplurality of different at-rest body orientations; receive the pluralityof image frames from the video camera; overlay one or more pixel lineson the plurality of image frames at a plurality of vertical positionsbased on a position of the patient, the one or more pixel linesdescribed as pixels at given horizontal positions within the pluralityof vertical positions of the plurality of image frames; scan the one ormore pixel lines at the plurality of vertical positions for luminancevalues; determine median horizontal positions of the one or more pixellines based on pixels having above average luminance within theluminance values; monitor the position of the patient on the supportsurface by determining which of the plurality of different at-rest bodyorientations the patient exhibits in the plurality of image frames byanalyzing the plurality of image frames based on the median horizontalpositions of the one or more pixel lines; determine the time in whichthe patient remains in a first patient position based on the monitoring,the first patient position corresponding to one of the plurality ofdifferent at-rest body orientations; issue a first alert within asurveillance window interface that indicates a correct patient positionand an indication that the patient is not in the correct patientposition based on the patient remaining in the first patient positionfor a duration exceeding one of the plurality of patient position timeintervals that is associated with the first patient position, thesurveillance window interface including time remaining in the correctpatient position, a next patient position according to the patientrepositioning schedule, and a field to submit a verification of acurrent position of the patient; detect a patient repositioning eventbased on the monitoring; determine whether the patient repositioningevent is not in compliance with the patient repositioning schedule basedon whether the repositioning event is an unscheduled event and theposition of the patient following the patient repositioning event is inconflict with the patient repositioning schedule; and issue a secondalert within the surveillance window interface that indicates thecorrect patient position and the indication that the patient is not inthe correct patient position based on the patient repositioning eventnot being in compliance with the patient repositioning schedule.
 14. Thesystem of claim 13, wherein the data processing system data isconfigured to detect the patient repositioning event by detecting atransition from a second patient position of the plurality of patientpositions to a third patient position of the plurality of patientpositions.
 15. The system of claim 13, wherein the patient repositioningschedule comprises an order for which the patient is to turn between theplurality of patient positions.
 16. The system of claim 15, wherein thedata processing system data is configured to determine whether thepatient repositioning event is not in compliance with the patientrepositioning schedule by determining whether the patient has turned outof the order.
 17. The system of claim 13, wherein the data processingsystem data is configured to determine whether the patient repositioningevent is not in compliance with the patient repositioning schedule bydetermining whether the patient has turned prematurely based on one ofthe plurality of patient position time intervals.
 18. The system ofclaim 13, wherein the data processing system data is configured to timestamp and store at least some of the plurality of image frames in thememory, wherein the patient is shown in each of the plurality of patientpositions in the at least some of the plurality of image frames.
 19. Thesystem of claim 13, wherein the data processing system data isconfigured to monitor the time in which the patient remains in the firstpatient position and detect the patient repositioning event by comparingat least one attribute of different image frames of the plurality ofimage frames between the different image frames.
 20. The system of claim13, wherein the plurality of patient positions comprise a left sideposition, a right side position, and a back side position.